| --- |
| library_name: transformers |
| license: other |
| base_model: google/gemma-3-1b-pt |
| tags: |
| - llama-factory |
| - full |
| - generated_from_trainer |
| model-index: |
| - name: gemma3-1B-pt |
| results: [] |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # gemma3-1B-pt |
|
|
| This model is a fine-tuned version of [google/gemma-3-1b-pt](https://huggingface.co/google/gemma-3-1b-pt) on the dolma_v17 dataset. |
| It achieves the following results on the evaluation set: |
| - Loss: 3.1214 |
| |
| ## Model description |
| |
| More information needed |
| |
| ## Intended uses & limitations |
| |
| More information needed |
| |
| ## Training and evaluation data |
| |
| More information needed |
| |
| ## Training procedure |
| |
| ### Training hyperparameters |
| |
| The following hyperparameters were used during training: |
| - learning_rate: 0.0003 |
| - train_batch_size: 4 |
| - eval_batch_size: 4 |
| - seed: 42 |
| - distributed_type: multi-GPU |
| - num_devices: 4 |
| - total_train_batch_size: 16 |
| - total_eval_batch_size: 16 |
| - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
| - lr_scheduler_type: cosine |
| - lr_scheduler_warmup_steps: 2000 |
| - training_steps: 480000 |
|
|
| ### Training results |
|
|
| | Training Loss | Epoch | Step | Validation Loss | |
| |:-------------:|:------:|:------:|:---------------:| |
| | 4.1231 | 0.0021 | 1000 | 3.5662 | |
| | 4.2873 | 0.0042 | 2000 | 3.9594 | |
| | 4.1824 | 0.0063 | 3000 | 3.9244 | |
| | 4.1178 | 0.0083 | 4000 | 3.8392 | |
| | 4.1489 | 0.0104 | 5000 | 3.7724 | |
| | 3.9733 | 0.0125 | 6000 | 3.7584 | |
| | 4.0348 | 0.0146 | 7000 | 3.7593 | |
| | 4.0383 | 0.0167 | 8000 | 3.7286 | |
| | 4.0088 | 0.0187 | 9000 | 3.6654 | |
| | 4.065 | 0.0208 | 10000 | 3.6618 | |
| | 3.9732 | 0.0229 | 11000 | 3.6465 | |
| | 3.9639 | 0.025 | 12000 | 3.6528 | |
| | 3.8999 | 0.0271 | 13000 | 3.6333 | |
| | 3.9205 | 0.0292 | 14000 | 3.6385 | |
| | 3.8237 | 0.0312 | 15000 | 3.6415 | |
| | 3.8754 | 0.0333 | 16000 | 3.6088 | |
| | 3.8118 | 0.0354 | 17000 | 3.5924 | |
| | 3.9038 | 0.0375 | 18000 | 3.5603 | |
| | 3.7947 | 0.0396 | 19000 | 3.5737 | |
| | 3.735 | 0.0417 | 20000 | 3.5600 | |
| | 3.8797 | 0.0437 | 21000 | 3.5516 | |
| | 3.7604 | 0.0458 | 22000 | 3.5607 | |
| | 3.7209 | 0.0479 | 23000 | 3.5560 | |
| | 3.8346 | 0.05 | 24000 | 3.5523 | |
| | 3.7705 | 0.0521 | 25000 | 3.5504 | |
| | 3.7442 | 0.0542 | 26000 | 3.5664 | |
| | 3.8061 | 0.0563 | 27000 | 3.5472 | |
| | 3.765 | 0.0583 | 28000 | 3.5457 | |
| | 3.7713 | 0.0604 | 29000 | 3.5490 | |
| | 3.7 | 0.0625 | 30000 | 3.5241 | |
| | 3.8156 | 0.0646 | 31000 | 3.5146 | |
| | 3.8258 | 0.0667 | 32000 | 3.5318 | |
| | 3.7444 | 0.0688 | 33000 | 3.5129 | |
| | 3.8583 | 0.0708 | 34000 | 3.5005 | |
| | 3.6806 | 0.0729 | 35000 | 3.5051 | |
| | 3.7325 | 0.075 | 36000 | 3.5013 | |
| | 3.7471 | 0.0771 | 37000 | 3.5093 | |
| | 3.7238 | 0.0792 | 38000 | 3.4953 | |
| | 3.6468 | 0.0813 | 39000 | 3.5099 | |
| | 3.6568 | 0.0833 | 40000 | 3.4913 | |
| | 3.6442 | 0.0854 | 41000 | 3.4889 | |
| | 3.6601 | 0.0875 | 42000 | 3.4442 | |
| | 3.6376 | 0.0896 | 43000 | 3.4345 | |
| | 3.7202 | 0.0917 | 44000 | 3.4283 | |
| | 3.7365 | 0.0938 | 45000 | 3.4413 | |
| | 3.6376 | 0.0958 | 46000 | 3.4199 | |
| | 3.7827 | 0.0979 | 47000 | 3.4258 | |
| | 3.6721 | 0.1 | 48000 | 3.4263 | |
| | 3.716 | 0.1021 | 49000 | 3.4274 | |
| | 3.6723 | 0.1042 | 50000 | 3.4224 | |
| | 3.655 | 0.1062 | 51000 | 3.4255 | |
| | 3.7093 | 0.1083 | 52000 | 3.4226 | |
| | 3.6237 | 0.1104 | 53000 | 3.4109 | |
| | 3.6489 | 0.1125 | 54000 | 3.4238 | |
| | 3.644 | 0.1146 | 55000 | 3.3997 | |
| | 3.6141 | 0.1167 | 56000 | 3.4242 | |
| | 3.7143 | 0.1187 | 57000 | 3.4003 | |
| | 3.5723 | 0.1208 | 58000 | 3.4008 | |
| | 3.604 | 0.1229 | 59000 | 3.4045 | |
| | 3.6356 | 0.125 | 60000 | 3.3880 | |
| | 3.6441 | 0.1271 | 61000 | 3.4103 | |
| | 3.6104 | 0.1292 | 62000 | 3.4073 | |
| | 3.5266 | 0.1313 | 63000 | 3.4091 | |
| | 3.6078 | 0.1333 | 64000 | 3.3986 | |
| | 3.5876 | 0.1354 | 65000 | 3.3921 | |
| | 3.5659 | 0.1375 | 66000 | 3.3873 | |
| | 3.6309 | 0.1396 | 67000 | 3.3915 | |
| | 3.6218 | 0.1417 | 68000 | 3.3934 | |
| | 3.6729 | 0.1437 | 69000 | 3.3975 | |
| | 3.639 | 0.1458 | 70000 | 3.3928 | |
| | 3.6776 | 0.1479 | 71000 | 3.3864 | |
| | 3.7602 | 0.15 | 72000 | 3.3904 | |
| | 3.6174 | 0.1521 | 73000 | 3.3774 | |
| | 3.5562 | 0.1542 | 74000 | 3.3821 | |
| | 3.5972 | 0.1562 | 75000 | 3.3636 | |
| | 3.6927 | 0.1583 | 76000 | 3.3814 | |
| | 3.7413 | 0.1604 | 77000 | 3.3923 | |
| | 3.6427 | 0.1625 | 78000 | 3.3948 | |
| | 3.622 | 0.1646 | 79000 | 3.3828 | |
| | 3.5513 | 0.1667 | 80000 | 3.3754 | |
| | 3.4515 | 0.1688 | 81000 | 3.3632 | |
| | 3.2936 | 0.1708 | 82000 | 3.3552 | |
| | 3.2549 | 0.1729 | 83000 | 3.3650 | |
| | 3.2611 | 0.175 | 84000 | 3.3684 | |
| | 3.3497 | 0.1771 | 85000 | 3.3693 | |
| | 3.2477 | 0.1792 | 86000 | 3.3722 | |
| | 3.2041 | 0.1812 | 87000 | 3.3734 | |
| | 3.2697 | 0.1833 | 88000 | 3.3873 | |
| | 3.2719 | 0.1854 | 89000 | 3.3667 | |
| | 3.2564 | 0.1875 | 90000 | 3.3574 | |
| | 3.2267 | 0.1896 | 91000 | 3.3660 | |
| | 3.2232 | 0.1917 | 92000 | 3.3628 | |
| | 3.2613 | 0.1938 | 93000 | 3.3590 | |
| | 3.2636 | 0.1958 | 94000 | 3.3611 | |
| | 3.2465 | 0.1979 | 95000 | 3.3565 | |
| | 3.2315 | 0.2 | 96000 | 3.3572 | |
| | 3.2118 | 0.2021 | 97000 | 3.3445 | |
| | 3.1693 | 0.2042 | 98000 | 3.3520 | |
| | 3.2664 | 0.2062 | 99000 | 3.3499 | |
| | 3.2739 | 0.2083 | 100000 | 3.3590 | |
| | 3.1831 | 0.2104 | 101000 | 3.3753 | |
| | 3.2189 | 0.2125 | 102000 | 3.3559 | |
| | 3.1941 | 0.2146 | 103000 | 3.3563 | |
| | 3.2745 | 0.2167 | 104000 | 3.3592 | |
| | 3.2005 | 0.2188 | 105000 | 3.3632 | |
| | 3.2354 | 0.2208 | 106000 | 3.3535 | |
| | 3.205 | 0.2229 | 107000 | 3.3498 | |
| | 3.2058 | 0.225 | 108000 | 3.3491 | |
| | 3.1774 | 0.2271 | 109000 | 3.3454 | |
| | 3.1795 | 0.2292 | 110000 | 3.3300 | |
| | 3.1168 | 0.2313 | 111000 | 3.3271 | |
| | 3.1848 | 0.2333 | 112000 | 3.3393 | |
| | 3.2029 | 0.2354 | 113000 | 3.3355 | |
| | 3.2201 | 0.2375 | 114000 | 3.3305 | |
| | 3.194 | 0.2396 | 115000 | 3.3400 | |
| | 3.143 | 0.2417 | 116000 | 3.3296 | |
| | 3.1782 | 0.2437 | 117000 | 3.3352 | |
| | 3.2255 | 0.2458 | 118000 | 3.3210 | |
| | 3.1641 | 0.2479 | 119000 | 3.3329 | |
| | 3.1812 | 0.25 | 120000 | 3.3350 | |
| | 3.1517 | 0.2521 | 121000 | 3.3229 | |
| | 3.1663 | 0.2542 | 122000 | 3.3097 | |
| | 3.1784 | 0.2562 | 123000 | 3.3108 | |
| | 3.2626 | 0.2583 | 124000 | 3.3117 | |
| | 3.1172 | 0.2604 | 125000 | 3.3113 | |
| | 3.1532 | 0.2625 | 126000 | 3.3107 | |
| | 3.1577 | 0.2646 | 127000 | 3.3154 | |
| | 3.1569 | 0.2667 | 128000 | 3.3134 | |
| | 3.16 | 0.2687 | 129000 | 3.3089 | |
| | 3.2018 | 0.2708 | 130000 | 3.3166 | |
| | 3.1327 | 0.2729 | 131000 | 3.3165 | |
| | 3.1258 | 0.275 | 132000 | 3.3044 | |
| | 3.1717 | 0.2771 | 133000 | 3.3030 | |
| | 3.0761 | 0.2792 | 134000 | 3.2902 | |
| | 3.1551 | 0.2812 | 135000 | 3.2997 | |
| | 3.1054 | 0.2833 | 136000 | 3.3039 | |
| | 3.1916 | 0.2854 | 137000 | 3.3056 | |
| | 3.144 | 0.2875 | 138000 | 3.3127 | |
| | 3.1609 | 0.2896 | 139000 | 3.3037 | |
| | 3.1749 | 0.2917 | 140000 | 3.3044 | |
| | 3.1346 | 0.2938 | 141000 | 3.3011 | |
| | 3.0608 | 0.2958 | 142000 | 3.2888 | |
| | 3.1251 | 0.2979 | 143000 | 3.2813 | |
| | 3.1208 | 0.3 | 144000 | 3.2812 | |
| | 3.0877 | 0.3021 | 145000 | 3.2931 | |
| | 3.0865 | 0.3042 | 146000 | 3.2948 | |
| | 3.1784 | 0.3063 | 147000 | 3.2959 | |
| | 3.1297 | 0.3083 | 148000 | 3.2894 | |
| | 3.1227 | 0.3104 | 149000 | 3.2777 | |
| | 3.1204 | 0.3125 | 150000 | 3.2952 | |
| | 3.0668 | 0.3146 | 151000 | 3.2917 | |
| | 3.2141 | 0.3167 | 152000 | 3.2860 | |
| | 3.0843 | 0.3187 | 153000 | 3.2891 | |
| | 3.0796 | 0.3208 | 154000 | 3.2822 | |
| | 3.1149 | 0.3229 | 155000 | 3.2855 | |
| | 3.0984 | 0.325 | 156000 | 3.2837 | |
| | 3.1173 | 0.3271 | 157000 | 3.2727 | |
| | 3.1454 | 0.3292 | 158000 | 3.2726 | |
| | 3.1479 | 0.3312 | 159000 | 3.2699 | |
| | 3.1113 | 0.3333 | 160000 | 3.2756 | |
| | 3.0758 | 0.3354 | 161000 | 3.2728 | |
| | 3.1652 | 0.3375 | 162000 | 3.2735 | |
| | 3.1214 | 0.3396 | 163000 | 3.2821 | |
| | 3.0668 | 0.3417 | 164000 | 3.2582 | |
| | 3.1123 | 0.3438 | 165000 | 3.2668 | |
| | 3.0657 | 0.3458 | 166000 | 3.2606 | |
| | 3.1118 | 0.3479 | 167000 | 3.2635 | |
| | 3.0879 | 0.35 | 168000 | 3.2460 | |
| | 3.1377 | 0.3521 | 169000 | 3.2582 | |
| | 3.1477 | 0.3542 | 170000 | 3.2647 | |
| | 3.1114 | 0.3563 | 171000 | 3.2594 | |
| | 3.1331 | 0.3583 | 172000 | 3.2698 | |
| | 3.0761 | 0.3604 | 173000 | 3.2682 | |
| | 3.0742 | 0.3625 | 174000 | 3.2621 | |
| | 3.0565 | 0.3646 | 175000 | 3.2494 | |
| | 3.1654 | 0.3667 | 176000 | 3.2576 | |
| | 3.1304 | 0.3688 | 177000 | 3.2636 | |
| | 3.1096 | 0.3708 | 178000 | 3.2633 | |
| | 3.1052 | 0.3729 | 179000 | 3.2796 | |
| | 3.0734 | 0.375 | 180000 | 3.2518 | |
| | 3.0673 | 0.3771 | 181000 | 3.2605 | |
| | 3.1222 | 0.3792 | 182000 | 3.2583 | |
| | 3.057 | 0.3812 | 183000 | 3.2566 | |
| | 3.1082 | 0.3833 | 184000 | 3.2596 | |
| | 3.0846 | 0.3854 | 185000 | 3.2570 | |
| | 3.0949 | 0.3875 | 186000 | 3.2472 | |
| | 3.1848 | 0.3896 | 187000 | 3.2504 | |
| | 3.0556 | 0.3917 | 188000 | 3.2502 | |
| | 3.0558 | 0.3937 | 189000 | 3.2365 | |
| | 3.1139 | 0.3958 | 190000 | 3.2431 | |
| | 3.0511 | 0.3979 | 191000 | 3.2405 | |
| | 3.1296 | 0.4 | 192000 | 3.2441 | |
| | 3.0968 | 0.4021 | 193000 | 3.2429 | |
| | 3.0972 | 0.4042 | 194000 | 3.2439 | |
| | 3.0497 | 0.4062 | 195000 | 3.2339 | |
| | 3.1313 | 0.4083 | 196000 | 3.2383 | |
| | 3.0513 | 0.4104 | 197000 | 3.2407 | |
| | 3.0183 | 0.4125 | 198000 | 3.2350 | |
| | 3.1177 | 0.4146 | 199000 | 3.2350 | |
| | 3.0348 | 0.4167 | 200000 | 3.2410 | |
| | 3.066 | 0.4188 | 201000 | 3.2402 | |
| | 3.0564 | 0.4208 | 202000 | 3.2386 | |
| | 3.106 | 0.4229 | 203000 | 3.2360 | |
| | 3.0704 | 0.425 | 204000 | 3.2313 | |
| | 3.063 | 0.4271 | 205000 | 3.2396 | |
| | 3.1267 | 0.4292 | 206000 | 3.2260 | |
| | 3.0825 | 0.4313 | 207000 | 3.2367 | |
| | 3.1342 | 0.4333 | 208000 | 3.2319 | |
| | 3.0563 | 0.4354 | 209000 | 3.2274 | |
| | 3.0054 | 0.4375 | 210000 | 3.2060 | |
| | 3.0109 | 0.4396 | 211000 | 3.2166 | |
| | 3.1039 | 0.4417 | 212000 | 3.2225 | |
| | 3.0479 | 0.4437 | 213000 | 3.2274 | |
| | 3.0666 | 0.4458 | 214000 | 3.2284 | |
| | 3.108 | 0.4479 | 215000 | 3.2263 | |
| | 3.0377 | 0.45 | 216000 | 3.2209 | |
| | 3.0507 | 0.4521 | 217000 | 3.2119 | |
| | 3.0592 | 0.4542 | 218000 | 3.2150 | |
| | 3.0199 | 0.4562 | 219000 | 3.2218 | |
| | 3.0404 | 0.4583 | 220000 | 3.2033 | |
| | 3.0542 | 0.4604 | 221000 | 3.2110 | |
| | 3.0055 | 0.4625 | 222000 | 3.2110 | |
| | 3.0923 | 0.4646 | 223000 | 3.2100 | |
| | 3.0718 | 0.4667 | 224000 | 3.2084 | |
| | 3.0477 | 0.4688 | 225000 | 3.2151 | |
| | 3.0587 | 0.4708 | 226000 | 3.2046 | |
| | 3.0412 | 0.4729 | 227000 | 3.2108 | |
| | 3.0504 | 0.475 | 228000 | 3.2118 | |
| | 3.0124 | 0.4771 | 229000 | 3.2043 | |
| | 3.0354 | 0.4792 | 230000 | 3.2057 | |
| | 3.0696 | 0.4813 | 231000 | 3.2128 | |
| | 3.0654 | 0.4833 | 232000 | 3.2110 | |
| | 3.0027 | 0.4854 | 233000 | 3.2003 | |
| | 3.0297 | 0.4875 | 234000 | 3.2057 | |
| | 3.051 | 0.4896 | 235000 | 3.2019 | |
| | 3.0446 | 0.4917 | 236000 | 3.1971 | |
| | 3.1235 | 0.4938 | 237000 | 3.1996 | |
| | 3.1057 | 0.4958 | 238000 | 3.1990 | |
| | 3.0277 | 0.4979 | 239000 | 3.1951 | |
| | 3.0262 | 0.5 | 240000 | 3.1884 | |
| | 3.0717 | 0.5021 | 241000 | 3.1836 | |
| | 3.0958 | 0.5042 | 242000 | 3.1968 | |
| | 3.1089 | 0.5062 | 243000 | 3.1826 | |
| | 3.0196 | 0.5083 | 244000 | 3.1879 | |
| | 3.0184 | 0.5104 | 245000 | 3.1874 | |
| | 3.0546 | 0.5125 | 246000 | 3.1896 | |
| | 3.0334 | 0.5146 | 247000 | 3.1843 | |
| | 3.0282 | 0.5167 | 248000 | 3.1836 | |
| | 2.9911 | 0.5188 | 249000 | 3.1830 | |
| | 3.0195 | 0.5208 | 250000 | 3.1847 | |
| | 3.0426 | 0.5229 | 251000 | 3.1859 | |
| | 3.0318 | 0.525 | 252000 | 3.1867 | |
| | 3.054 | 0.5271 | 253000 | 3.1793 | |
| | 3.0173 | 0.5292 | 254000 | 3.1803 | |
| | 3.0155 | 0.5312 | 255000 | 3.1785 | |
| | 3.1029 | 0.5333 | 256000 | 3.1764 | |
| | 3.0426 | 0.5354 | 257000 | 3.1878 | |
| | 3.012 | 0.5375 | 258000 | 3.1930 | |
| | 3.0191 | 0.5396 | 259000 | 3.1967 | |
| | 2.9717 | 0.5417 | 260000 | 3.1993 | |
| | 3.0294 | 0.5437 | 261000 | 3.1918 | |
| | 2.9724 | 0.5458 | 262000 | 3.1919 | |
| | 2.9895 | 0.5479 | 263000 | 3.1855 | |
| | 3.0021 | 0.55 | 264000 | 3.1869 | |
| | 3.072 | 0.5521 | 265000 | 3.1755 | |
| | 3.0339 | 0.5542 | 266000 | 3.1803 | |
| | 3.0421 | 0.5563 | 267000 | 3.1872 | |
| | 2.997 | 0.5583 | 268000 | 3.1742 | |
| | 3.0436 | 0.5604 | 269000 | 3.1745 | |
| | 3.0852 | 0.5625 | 270000 | 3.1764 | |
| | 3.0321 | 0.5646 | 271000 | 3.1642 | |
| | 3.1182 | 0.5667 | 272000 | 3.1627 | |
| | 2.9925 | 0.5687 | 273000 | 3.1729 | |
| | 3.0205 | 0.5708 | 274000 | 3.1610 | |
| | 3.0018 | 0.5729 | 275000 | 3.1760 | |
| | 3.0344 | 0.575 | 276000 | 3.1704 | |
| | 2.995 | 0.5771 | 277000 | 3.1677 | |
| | 3.0158 | 0.5792 | 278000 | 3.1628 | |
| | 2.3228 | 0.5813 | 279000 | 3.1738 | |
| | 1.9767 | 0.5833 | 280000 | 3.1792 | |
| | 1.7793 | 0.5854 | 281000 | 3.1928 | |
| | 1.6275 | 0.5875 | 282000 | 3.2050 | |
| | 1.5627 | 0.5896 | 283000 | 3.2088 | |
| | 1.5523 | 0.5917 | 284000 | 3.2137 | |
| | 1.6109 | 0.5938 | 285000 | 3.2236 | |
| | 1.5898 | 0.5958 | 286000 | 3.2250 | |
| | 1.6009 | 0.5979 | 287000 | 3.2402 | |
| | 1.6127 | 0.6 | 288000 | 3.2470 | |
| | 1.5782 | 0.6021 | 289000 | 3.2489 | |
| | 1.5149 | 0.6042 | 290000 | 3.2529 | |
| | 1.4487 | 0.6062 | 291000 | 3.2520 | |
| | 1.6001 | 0.6083 | 292000 | 3.2613 | |
| | 1.4521 | 0.6104 | 293000 | 3.2601 | |
| | 1.4562 | 0.6125 | 294000 | 3.2719 | |
| | 1.4204 | 0.6146 | 295000 | 3.2696 | |
| | 1.6124 | 0.6167 | 296000 | 3.2667 | |
| | 1.4523 | 0.6188 | 297000 | 3.2755 | |
| | 1.4082 | 0.6208 | 298000 | 3.2837 | |
| | 1.4225 | 0.6229 | 299000 | 3.2795 | |
| | 1.3266 | 0.625 | 300000 | 3.2911 | |
| | 1.5021 | 0.6271 | 301000 | 3.2850 | |
| | 1.3539 | 0.6292 | 302000 | 3.2820 | |
| | 1.3693 | 0.6312 | 303000 | 3.2864 | |
| | 1.3012 | 0.6333 | 304000 | 3.2869 | |
| | 1.4507 | 0.6354 | 305000 | 3.2953 | |
| | 1.4535 | 0.6375 | 306000 | 3.2987 | |
| | 2.2856 | 0.6396 | 307000 | 3.2699 | |
| | 2.7748 | 0.6417 | 308000 | 3.2334 | |
| | 2.9475 | 0.6438 | 309000 | 3.2104 | |
| | 3.0064 | 0.6458 | 310000 | 3.2035 | |
| | 2.9668 | 0.6479 | 311000 | 3.1969 | |
| | 3.0531 | 0.65 | 312000 | 3.1961 | |
| | 3.0614 | 0.6521 | 313000 | 3.1923 | |
| | 3.0591 | 0.6542 | 314000 | 3.1943 | |
| | 3.0073 | 0.6562 | 315000 | 3.1867 | |
| | 2.9947 | 0.6583 | 316000 | 3.1746 | |
| | 2.9789 | 0.6604 | 317000 | 3.1735 | |
| | 3.0405 | 0.6625 | 318000 | 3.1759 | |
| | 3.054 | 0.6646 | 319000 | 3.1717 | |
| | 3.018 | 0.6667 | 320000 | 3.1752 | |
| | 2.9991 | 0.6687 | 321000 | 3.1736 | |
| | 3.0098 | 0.6708 | 322000 | 3.1692 | |
| | 2.9798 | 0.6729 | 323000 | 3.1703 | |
| | 3.0329 | 0.675 | 324000 | 3.1673 | |
| | 2.9803 | 0.6771 | 325000 | 3.1661 | |
| | 2.9593 | 0.6792 | 326000 | 3.1637 | |
| | 3.0197 | 0.6813 | 327000 | 3.1622 | |
| | 3.0118 | 0.6833 | 328000 | 3.1563 | |
| | 2.912 | 0.6854 | 329000 | 3.1587 | |
| | 2.9475 | 0.6875 | 330000 | 3.1502 | |
| | 2.993 | 0.6896 | 331000 | 3.1630 | |
| | 3.058 | 0.6917 | 332000 | 3.1605 | |
| | 2.9426 | 0.6937 | 333000 | 3.1618 | |
| | 2.9845 | 0.6958 | 334000 | 3.1599 | |
| | 2.9986 | 0.6979 | 335000 | 3.1524 | |
| | 3.0294 | 0.7 | 336000 | 3.1511 | |
| | 2.9993 | 0.7021 | 337000 | 3.1524 | |
| | 3.0504 | 0.7042 | 338000 | 3.1552 | |
| | 2.9536 | 0.7063 | 339000 | 3.1461 | |
| | 2.965 | 0.7083 | 340000 | 3.1518 | |
| | 3.0436 | 0.7104 | 341000 | 3.1478 | |
| | 2.9228 | 0.7125 | 342000 | 3.1497 | |
| | 2.9819 | 0.7146 | 343000 | 3.1413 | |
| | 2.9757 | 0.7167 | 344000 | 3.1497 | |
| | 2.9522 | 0.7188 | 345000 | 3.1525 | |
| | 2.9428 | 0.7208 | 346000 | 3.1517 | |
| | 2.9761 | 0.7229 | 347000 | 3.1484 | |
| | 2.8803 | 0.725 | 348000 | 3.1525 | |
| | 2.9354 | 0.7271 | 349000 | 3.1458 | |
| | 3.0103 | 0.7292 | 350000 | 3.1408 | |
| | 2.9791 | 0.7312 | 351000 | 3.1399 | |
| | 2.9398 | 0.7333 | 352000 | 3.1407 | |
| | 3.0778 | 0.7354 | 353000 | 3.1378 | |
| | 2.9817 | 0.7375 | 354000 | 3.1464 | |
| | 3.0106 | 0.7396 | 355000 | 3.1457 | |
| | 2.9918 | 0.7417 | 356000 | 3.1448 | |
| | 2.9874 | 0.7438 | 357000 | 3.1443 | |
| | 2.9971 | 0.7458 | 358000 | 3.1458 | |
| | 3.0022 | 0.7479 | 359000 | 3.1436 | |
| | 3.004 | 0.75 | 360000 | 3.1378 | |
| | 3.0036 | 0.7521 | 361000 | 3.1368 | |
| | 2.9601 | 0.7542 | 362000 | 3.1342 | |
| | 2.999 | 0.7562 | 363000 | 3.1400 | |
| | 2.9794 | 0.7583 | 364000 | 3.1344 | |
| | 2.9638 | 0.7604 | 365000 | 3.1393 | |
| | 3.0029 | 0.7625 | 366000 | 3.1300 | |
| | 3.0533 | 0.7646 | 367000 | 3.1304 | |
| | 3.0585 | 0.7667 | 368000 | 3.1325 | |
| | 3.0032 | 0.7688 | 369000 | 3.1388 | |
| | 3.0029 | 0.7708 | 370000 | 3.1375 | |
| | 3.0661 | 0.7729 | 371000 | 3.1367 | |
| | 3.012 | 0.775 | 372000 | 3.1381 | |
| | 2.9817 | 0.7771 | 373000 | 3.1265 | |
| | 2.9661 | 0.7792 | 374000 | 3.1262 | |
| | 3.0085 | 0.7812 | 375000 | 3.1233 | |
| | 2.9662 | 0.7833 | 376000 | 3.1351 | |
| | 2.9988 | 0.7854 | 377000 | 3.1293 | |
| | 2.9812 | 0.7875 | 378000 | 3.1308 | |
| | 3.0652 | 0.7896 | 379000 | 3.1286 | |
| | 3.0235 | 0.7917 | 380000 | 3.1304 | |
| | 3.0019 | 0.7937 | 381000 | 3.1268 | |
| | 3.01 | 0.7958 | 382000 | 3.1255 | |
| | 2.9505 | 0.7979 | 383000 | 3.1333 | |
| | 2.9493 | 0.8 | 384000 | 3.1215 | |
| | 2.9672 | 0.8021 | 385000 | 3.1309 | |
| | 3.0173 | 0.8042 | 386000 | 3.1319 | |
| | 2.9647 | 0.8063 | 387000 | 3.1254 | |
| | 3.0428 | 0.8083 | 388000 | 3.1230 | |
| | 3.0167 | 0.8104 | 389000 | 3.1234 | |
| | 2.9247 | 0.8125 | 390000 | 3.1240 | |
| | 2.9828 | 0.8146 | 391000 | 3.1297 | |
| | 2.9665 | 0.8167 | 392000 | 3.1284 | |
| | 3.0186 | 0.8187 | 393000 | 3.1222 | |
| | 3.0044 | 0.8208 | 394000 | 3.1213 | |
| | 2.9925 | 0.8229 | 395000 | 3.1142 | |
| | 2.9604 | 0.825 | 396000 | 3.1282 | |
| | 3.0043 | 0.8271 | 397000 | 3.1274 | |
| | 2.926 | 0.8292 | 398000 | 3.1257 | |
| | 2.9929 | 0.8313 | 399000 | 3.1256 | |
| | 3.0688 | 0.8333 | 400000 | 3.1265 | |
| | 3.0416 | 0.8354 | 401000 | 3.1266 | |
| | 3.0489 | 0.8375 | 402000 | 3.1259 | |
| | 2.9436 | 0.8396 | 403000 | 3.1281 | |
| | 2.9727 | 0.8417 | 404000 | 3.1234 | |
| | 2.9545 | 0.8438 | 405000 | 3.1255 | |
| | 2.9414 | 0.8458 | 406000 | 3.1325 | |
| | 3.0126 | 0.8479 | 407000 | 3.1268 | |
| | 3.1509 | 0.85 | 408000 | 3.1238 | |
| | 3.3038 | 0.8521 | 409000 | 3.1199 | |
| | 3.3277 | 0.8542 | 410000 | 3.1228 | |
| | 3.3874 | 0.8562 | 411000 | 3.1185 | |
| | 3.3156 | 0.8583 | 412000 | 3.1190 | |
| | 3.3459 | 0.8604 | 413000 | 3.1105 | |
| | 3.31 | 0.8625 | 414000 | 3.1084 | |
| | 3.5069 | 0.8646 | 415000 | 3.1131 | |
| | 3.4144 | 0.8667 | 416000 | 3.1155 | |
| | 3.3535 | 0.8688 | 417000 | 3.1152 | |
| | 3.3127 | 0.8708 | 418000 | 3.1133 | |
| | 3.2963 | 0.8729 | 419000 | 3.1199 | |
| | 3.3968 | 0.875 | 420000 | 3.1085 | |
| | 3.3216 | 0.8771 | 421000 | 3.1200 | |
| | 3.4374 | 0.8792 | 422000 | 3.1195 | |
| | 3.3227 | 0.8812 | 423000 | 3.1181 | |
| | 3.3693 | 0.8833 | 424000 | 3.1137 | |
| | 3.4757 | 0.8854 | 425000 | 3.1174 | |
| | 3.4784 | 0.8875 | 426000 | 3.1190 | |
| | 3.4194 | 0.8896 | 427000 | 3.1173 | |
| | 3.4026 | 0.8917 | 428000 | 3.1163 | |
| | 3.3905 | 0.8938 | 429000 | 3.1225 | |
| | 3.3835 | 0.8958 | 430000 | 3.1205 | |
| | 3.4056 | 0.8979 | 431000 | 3.1231 | |
| | 3.3465 | 0.9 | 432000 | 3.1249 | |
| | 3.4562 | 0.9021 | 433000 | 3.1224 | |
| | 3.4784 | 0.9042 | 434000 | 3.1248 | |
| | 3.406 | 0.9062 | 435000 | 3.1175 | |
| | 3.4705 | 0.9083 | 436000 | 3.1249 | |
| | 3.4704 | 0.9104 | 437000 | 3.1206 | |
| | 3.5798 | 0.9125 | 438000 | 3.1205 | |
| | 3.4384 | 0.9146 | 439000 | 3.1252 | |
| | 3.3253 | 0.9167 | 440000 | 3.1265 | |
| | 3.4023 | 0.9187 | 441000 | 3.1197 | |
| | 3.3925 | 0.9208 | 442000 | 3.1198 | |
| | 3.4729 | 0.9229 | 443000 | 3.1293 | |
| | 3.4435 | 0.925 | 444000 | 3.1171 | |
| | 3.3504 | 0.9271 | 445000 | 3.1251 | |
| | 3.3285 | 0.9292 | 446000 | 3.1254 | |
| | 3.4275 | 0.9313 | 447000 | 3.1264 | |
| | 3.4027 | 0.9333 | 448000 | 3.1234 | |
| | 3.5068 | 0.9354 | 449000 | 3.1182 | |
| | 3.3969 | 0.9375 | 450000 | 3.1248 | |
| | 3.3506 | 0.9396 | 451000 | 3.1235 | |
| | 3.3393 | 0.9417 | 452000 | 3.1292 | |
| | 3.4663 | 0.9437 | 453000 | 3.1264 | |
| | 3.3814 | 0.9458 | 454000 | 3.1288 | |
| | 3.4535 | 0.9479 | 455000 | 3.1257 | |
| | 3.3788 | 0.95 | 456000 | 3.1198 | |
| | 3.4481 | 0.9521 | 457000 | 3.1267 | |
| | 3.357 | 0.9542 | 458000 | 3.1249 | |
| | 3.3328 | 0.9563 | 459000 | 3.1227 | |
| | 3.3335 | 0.9583 | 460000 | 3.1249 | |
| | 3.4672 | 0.9604 | 461000 | 3.1267 | |
| | 3.3894 | 0.9625 | 462000 | 3.1250 | |
| | 3.4675 | 0.9646 | 463000 | 3.1243 | |
| | 3.4096 | 0.9667 | 464000 | 3.1296 | |
| | 3.4812 | 0.9688 | 465000 | 3.1171 | |
| | 3.3559 | 0.9708 | 466000 | 3.1278 | |
| | 3.4049 | 0.9729 | 467000 | 3.1252 | |
| | 3.5119 | 0.975 | 468000 | 3.1271 | |
| | 3.4627 | 0.9771 | 469000 | 3.1241 | |
| | 3.4584 | 0.9792 | 470000 | 3.1247 | |
| | 3.467 | 0.9812 | 471000 | 3.1251 | |
| | 3.492 | 0.9833 | 472000 | 3.1183 | |
| | 3.3404 | 0.9854 | 473000 | 3.1257 | |
| | 3.4141 | 0.9875 | 474000 | 3.1280 | |
| | 3.4339 | 0.9896 | 475000 | 3.1288 | |
| | 3.3863 | 0.9917 | 476000 | 3.1263 | |
| | 3.3718 | 0.9938 | 477000 | 3.1228 | |
| | 3.4157 | 0.9958 | 478000 | 3.1177 | |
| | 3.3245 | 0.9979 | 479000 | 3.1292 | |
| | 3.4154 | 1.0 | 480000 | 3.1214 | |
|
|
|
|
| ### Framework versions |
|
|
| - Transformers 4.50.3 |
| - Pytorch 2.6.0+cu124 |
| - Datasets 2.21.0 |
| - Tokenizers 0.21.4 |
|
|
