| --- |
| library_name: transformers |
| license: other |
| base_model: meta-llama/Llama-3.2-1B |
| tags: |
| - llama-factory |
| - full |
| - generated_from_trainer |
| model-index: |
| - name: llama3.2-1B-pt |
| results: [] |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # llama3.2-1B-pt |
|
|
| This model is a fine-tuned version of [meta-llama/Llama-3.2-1B](https://huggingface.co/meta-llama/Llama-3.2-1B) on the dolma_v17 dataset. |
| It achieves the following results on the evaluation set: |
| - Loss: 3.3814 |
| |
| ## Model description |
| |
| More information needed |
| |
| ## Intended uses & limitations |
| |
| More information needed |
| |
| ## Training and evaluation data |
| |
| More information needed |
| |
| ## Training procedure |
| |
| ### Training hyperparameters |
| |
| The following hyperparameters were used during training: |
| - learning_rate: 0.0003 |
| - train_batch_size: 4 |
| - eval_batch_size: 4 |
| - seed: 42 |
| - distributed_type: multi-GPU |
| - num_devices: 4 |
| - total_train_batch_size: 16 |
| - total_eval_batch_size: 16 |
| - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
| - lr_scheduler_type: cosine |
| - lr_scheduler_warmup_steps: 2000 |
| - training_steps: 480000 |
|
|
| ### Training results |
|
|
| | Training Loss | Epoch | Step | Validation Loss | |
| |:-------------:|:------:|:------:|:---------------:| |
| | 4.141 | 0.0021 | 1000 | 4.0261 | |
| | 4.5266 | 0.0042 | 2000 | 4.5481 | |
| | 4.59 | 0.0063 | 3000 | 4.5945 | |
| | 4.4559 | 0.0083 | 4000 | 4.4359 | |
| | 4.4322 | 0.0104 | 5000 | 4.3558 | |
| | 4.2777 | 0.0125 | 6000 | 4.2885 | |
| | 4.2747 | 0.0146 | 7000 | 4.2384 | |
| | 4.2632 | 0.0167 | 8000 | 4.2193 | |
| | 4.2457 | 0.0187 | 9000 | 4.1833 | |
| | 4.1475 | 0.0208 | 10000 | 4.1386 | |
| | 4.141 | 0.0229 | 11000 | 4.0980 | |
| | 4.1236 | 0.025 | 12000 | 4.0966 | |
| | 4.1118 | 0.0271 | 13000 | 4.0820 | |
| | 4.1394 | 0.0292 | 14000 | 4.0277 | |
| | 4.006 | 0.0312 | 15000 | 4.0100 | |
| | 4.0928 | 0.0333 | 16000 | 4.0256 | |
| | 4.0917 | 0.0354 | 17000 | 4.0059 | |
| | 4.0418 | 0.0375 | 18000 | 3.9906 | |
| | 4.069 | 0.0396 | 19000 | 3.9773 | |
| | 3.9632 | 0.0417 | 20000 | 3.9625 | |
| | 4.1044 | 0.0437 | 21000 | 3.9744 | |
| | 4.0262 | 0.0458 | 22000 | 3.9423 | |
| | 4.0516 | 0.0479 | 23000 | 3.9458 | |
| | 4.0737 | 0.05 | 24000 | 3.9307 | |
| | 3.9643 | 0.0521 | 25000 | 3.9314 | |
| | 3.9472 | 0.0542 | 26000 | 3.9337 | |
| | 3.9465 | 0.0563 | 27000 | 3.9346 | |
| | 3.902 | 0.0583 | 28000 | 3.9161 | |
| | 4.0343 | 0.0604 | 29000 | 3.8905 | |
| | 3.9185 | 0.0625 | 30000 | 3.8983 | |
| | 3.8769 | 0.0646 | 31000 | 3.8749 | |
| | 3.9741 | 0.0667 | 32000 | 3.8434 | |
| | 3.8712 | 0.0688 | 33000 | 3.8822 | |
| | 3.7466 | 0.0708 | 34000 | 3.8275 | |
| | 3.9764 | 0.0729 | 35000 | 3.8520 | |
| | 3.8212 | 0.075 | 36000 | 3.8515 | |
| | 3.8671 | 0.0771 | 37000 | 3.8439 | |
| | 3.8405 | 0.0792 | 38000 | 3.8555 | |
| | 3.8123 | 0.0813 | 39000 | 3.8229 | |
| | 3.9641 | 0.0833 | 40000 | 3.8233 | |
| | 3.7526 | 0.0854 | 41000 | 3.8344 | |
| | 3.9086 | 0.0875 | 42000 | 3.8212 | |
| | 3.8417 | 0.0896 | 43000 | 3.8113 | |
| | 3.8525 | 0.0917 | 44000 | 3.7934 | |
| | 3.7642 | 0.0938 | 45000 | 3.8093 | |
| | 3.8702 | 0.0958 | 46000 | 3.7896 | |
| | 3.8191 | 0.0979 | 47000 | 3.7950 | |
| | 3.8887 | 0.1 | 48000 | 3.7916 | |
| | 3.8501 | 0.1021 | 49000 | 3.7954 | |
| | 3.7836 | 0.1042 | 50000 | 3.7698 | |
| | 3.8468 | 0.1062 | 51000 | 3.7577 | |
| | 3.8272 | 0.1083 | 52000 | 3.7868 | |
| | 3.8116 | 0.1104 | 53000 | 3.7769 | |
| | 3.7229 | 0.1125 | 54000 | 3.7917 | |
| | 3.7081 | 0.1146 | 55000 | 3.7711 | |
| | 3.8381 | 0.1167 | 56000 | 3.7697 | |
| | 3.7386 | 0.1187 | 57000 | 3.7559 | |
| | 3.75 | 0.1208 | 58000 | 3.7578 | |
| | 3.7611 | 0.1229 | 59000 | 3.7471 | |
| | 3.6357 | 0.125 | 60000 | 3.7345 | |
| | 3.8717 | 0.1271 | 61000 | 3.7425 | |
| | 3.7516 | 0.1292 | 62000 | 3.7501 | |
| | 3.6488 | 0.1313 | 63000 | 3.7434 | |
| | 3.7167 | 0.1333 | 64000 | 3.7177 | |
| | 3.6957 | 0.1354 | 65000 | 3.7139 | |
| | 3.693 | 0.1375 | 66000 | 3.7310 | |
| | 3.8045 | 0.1396 | 67000 | 3.7135 | |
| | 3.8231 | 0.1417 | 68000 | 3.7090 | |
| | 3.7704 | 0.1437 | 69000 | 3.7081 | |
| | 3.7848 | 0.1458 | 70000 | 3.7024 | |
| | 3.6931 | 0.1479 | 71000 | 3.6950 | |
| | 3.8635 | 0.15 | 72000 | 3.6928 | |
| | 3.8062 | 0.1521 | 73000 | 3.6915 | |
| | 3.6754 | 0.1542 | 74000 | 3.7145 | |
| | 3.8106 | 0.1562 | 75000 | 3.6927 | |
| | 3.7576 | 0.1583 | 76000 | 3.7038 | |
| | 3.7289 | 0.1604 | 77000 | 3.6871 | |
| | 3.7628 | 0.1625 | 78000 | 3.6730 | |
| | 3.6676 | 0.1646 | 79000 | 3.6735 | |
| | 3.5012 | 0.1667 | 80000 | 3.6606 | |
| | 3.4192 | 0.1688 | 81000 | 3.6842 | |
| | 3.4183 | 0.1708 | 82000 | 3.6978 | |
| | 3.4288 | 0.1729 | 83000 | 3.6978 | |
| | 3.3554 | 0.175 | 84000 | 3.6865 | |
| | 3.375 | 0.1771 | 85000 | 3.6816 | |
| | 3.3618 | 0.1792 | 86000 | 3.6732 | |
| | 3.4111 | 0.1812 | 87000 | 3.6747 | |
| | 3.4076 | 0.1833 | 88000 | 3.6576 | |
| | 3.321 | 0.1854 | 89000 | 3.6939 | |
| | 3.3093 | 0.1875 | 90000 | 3.6796 | |
| | 3.3298 | 0.1896 | 91000 | 3.6907 | |
| | 3.2924 | 0.1917 | 92000 | 3.6911 | |
| | 3.383 | 0.1938 | 93000 | 3.6845 | |
| | 3.3564 | 0.1958 | 94000 | 3.6801 | |
| | 3.3192 | 0.1979 | 95000 | 3.6735 | |
| | 3.3107 | 0.2 | 96000 | 3.6852 | |
| | 3.3519 | 0.2021 | 97000 | 3.6796 | |
| | 3.3198 | 0.2042 | 98000 | 3.6680 | |
| | 3.2356 | 0.2062 | 99000 | 3.6669 | |
| | 3.3781 | 0.2083 | 100000 | 3.6826 | |
| | 3.3048 | 0.2104 | 101000 | 3.6540 | |
| | 3.2758 | 0.2125 | 102000 | 3.6481 | |
| | 3.3009 | 0.2146 | 103000 | 3.6656 | |
| | 3.3017 | 0.2167 | 104000 | 3.6755 | |
| | 3.2664 | 0.2188 | 105000 | 3.6714 | |
| | 3.3034 | 0.2208 | 106000 | 3.6694 | |
| | 3.2943 | 0.2229 | 107000 | 3.6815 | |
| | 3.293 | 0.225 | 108000 | 3.6753 | |
| | 3.3475 | 0.2271 | 109000 | 3.6721 | |
| | 3.3305 | 0.2292 | 110000 | 3.6722 | |
| | 3.3044 | 0.2313 | 111000 | 3.6733 | |
| | 3.3002 | 0.2333 | 112000 | 3.6798 | |
| | 3.2265 | 0.2354 | 113000 | 3.6611 | |
| | 3.2664 | 0.2375 | 114000 | 3.6662 | |
| | 3.2878 | 0.2396 | 115000 | 3.6675 | |
| | 3.2847 | 0.2417 | 116000 | 3.6852 | |
| | 3.2496 | 0.2437 | 117000 | 3.6813 | |
| | 3.281 | 0.2458 | 118000 | 3.6718 | |
| | 3.2697 | 0.2479 | 119000 | 3.6719 | |
| | 3.2208 | 0.25 | 120000 | 3.6595 | |
| | 3.2665 | 0.2521 | 121000 | 3.6706 | |
| | 3.265 | 0.2542 | 122000 | 3.6546 | |
| | 3.3039 | 0.2562 | 123000 | 3.6668 | |
| | 3.2742 | 0.2583 | 124000 | 3.6628 | |
| | 3.2607 | 0.2604 | 125000 | 3.6577 | |
| | 3.2918 | 0.2625 | 126000 | 3.6540 | |
| | 3.2361 | 0.2646 | 127000 | 3.6403 | |
| | 3.2914 | 0.2667 | 128000 | 3.6429 | |
| | 3.2049 | 0.2687 | 129000 | 3.6321 | |
| | 3.2311 | 0.2708 | 130000 | 3.6408 | |
| | 3.2329 | 0.2729 | 131000 | 3.6315 | |
| | 3.2318 | 0.275 | 132000 | 3.6359 | |
| | 3.2287 | 0.2771 | 133000 | 3.6353 | |
| | 3.2889 | 0.2792 | 134000 | 3.6314 | |
| | 3.262 | 0.2812 | 135000 | 3.6412 | |
| | 3.2409 | 0.2833 | 136000 | 3.6325 | |
| | 3.2604 | 0.2854 | 137000 | 3.6292 | |
| | 3.1974 | 0.2875 | 138000 | 3.6217 | |
| | 3.2136 | 0.2896 | 139000 | 3.6326 | |
| | 3.2064 | 0.2917 | 140000 | 3.6171 | |
| | 3.2019 | 0.2938 | 141000 | 3.6307 | |
| | 3.2675 | 0.2958 | 142000 | 3.6220 | |
| | 3.2218 | 0.2979 | 143000 | 3.6384 | |
| | 3.232 | 0.3 | 144000 | 3.6349 | |
| | 3.2268 | 0.3021 | 145000 | 3.6209 | |
| | 3.2527 | 0.3042 | 146000 | 3.6154 | |
| | 3.3142 | 0.3063 | 147000 | 3.6200 | |
| | 3.2486 | 0.3083 | 148000 | 3.6195 | |
| | 3.1876 | 0.3104 | 149000 | 3.6249 | |
| | 3.2235 | 0.3125 | 150000 | 3.6284 | |
| | 3.2296 | 0.3146 | 151000 | 3.6220 | |
| | 3.2066 | 0.3167 | 152000 | 3.6319 | |
| | 3.2517 | 0.3187 | 153000 | 3.6236 | |
| | 3.1939 | 0.3208 | 154000 | 3.6191 | |
| | 3.2645 | 0.3229 | 155000 | 3.6178 | |
| | 3.2388 | 0.325 | 156000 | 3.6212 | |
| | 3.1929 | 0.3271 | 157000 | 3.6181 | |
| | 3.1655 | 0.3292 | 158000 | 3.6202 | |
| | 3.1706 | 0.3312 | 159000 | 3.6174 | |
| | 3.2138 | 0.3333 | 160000 | 3.6020 | |
| | 3.2086 | 0.3354 | 161000 | 3.6076 | |
| | 3.2253 | 0.3375 | 162000 | 3.6035 | |
| | 3.2249 | 0.3396 | 163000 | 3.5919 | |
| | 3.1773 | 0.3417 | 164000 | 3.5994 | |
| | 3.1898 | 0.3438 | 165000 | 3.5999 | |
| | 3.2367 | 0.3458 | 166000 | 3.5893 | |
| | 3.1909 | 0.3479 | 167000 | 3.6001 | |
| | 3.158 | 0.35 | 168000 | 3.6071 | |
| | 3.2281 | 0.3521 | 169000 | 3.5893 | |
| | 3.2258 | 0.3542 | 170000 | 3.5900 | |
| | 3.1549 | 0.3563 | 171000 | 3.5793 | |
| | 3.1841 | 0.3583 | 172000 | 3.5745 | |
| | 3.1991 | 0.3604 | 173000 | 3.5817 | |
| | 3.1658 | 0.3625 | 174000 | 3.5939 | |
| | 3.1957 | 0.3646 | 175000 | 3.5897 | |
| | 3.1929 | 0.3667 | 176000 | 3.5836 | |
| | 3.1652 | 0.3688 | 177000 | 3.5806 | |
| | 3.2392 | 0.3708 | 178000 | 3.5756 | |
| | 3.2093 | 0.3729 | 179000 | 3.5837 | |
| | 3.1547 | 0.375 | 180000 | 3.5854 | |
| | 3.2009 | 0.3771 | 181000 | 3.5779 | |
| | 3.2153 | 0.3792 | 182000 | 3.5743 | |
| | 3.2005 | 0.3812 | 183000 | 3.5732 | |
| | 3.1961 | 0.3833 | 184000 | 3.5855 | |
| | 3.1965 | 0.3854 | 185000 | 3.5874 | |
| | 3.143 | 0.3875 | 186000 | 3.5737 | |
| | 3.1589 | 0.3896 | 187000 | 3.5757 | |
| | 3.1506 | 0.3917 | 188000 | 3.5729 | |
| | 3.1764 | 0.3937 | 189000 | 3.5752 | |
| | 3.1561 | 0.3958 | 190000 | 3.5613 | |
| | 3.2287 | 0.3979 | 191000 | 3.5737 | |
| | 3.1566 | 0.4 | 192000 | 3.5636 | |
| | 3.2028 | 0.4021 | 193000 | 3.5612 | |
| | 3.1997 | 0.4042 | 194000 | 3.5643 | |
| | 3.1413 | 0.4062 | 195000 | 3.5683 | |
| | 3.1797 | 0.4083 | 196000 | 3.5553 | |
| | 3.2138 | 0.4104 | 197000 | 3.5717 | |
| | 3.207 | 0.4125 | 198000 | 3.5605 | |
| | 3.1739 | 0.4146 | 199000 | 3.5592 | |
| | 3.1325 | 0.4167 | 200000 | 3.5606 | |
| | 3.1089 | 0.4188 | 201000 | 3.5621 | |
| | 3.1753 | 0.4208 | 202000 | 3.5534 | |
| | 3.1524 | 0.4229 | 203000 | 3.5476 | |
| | 3.1589 | 0.425 | 204000 | 3.5586 | |
| | 3.1351 | 0.4271 | 205000 | 3.5490 | |
| | 3.1081 | 0.4292 | 206000 | 3.5423 | |
| | 3.0467 | 0.4313 | 207000 | 3.5523 | |
| | 3.1734 | 0.4333 | 208000 | 3.5477 | |
| | 3.1146 | 0.4354 | 209000 | 3.5477 | |
| | 3.151 | 0.4375 | 210000 | 3.5322 | |
| | 3.1045 | 0.4396 | 211000 | 3.5322 | |
| | 3.1233 | 0.4417 | 212000 | 3.5326 | |
| | 3.1608 | 0.4437 | 213000 | 3.5327 | |
| | 3.128 | 0.4458 | 214000 | 3.5458 | |
| | 3.1645 | 0.4479 | 215000 | 3.5342 | |
| | 3.2083 | 0.45 | 216000 | 3.5280 | |
| | 3.0882 | 0.4521 | 217000 | 3.5398 | |
| | 3.1598 | 0.4542 | 218000 | 3.5412 | |
| | 3.1839 | 0.4562 | 219000 | 3.5390 | |
| | 3.1684 | 0.4583 | 220000 | 3.5365 | |
| | 3.1318 | 0.4604 | 221000 | 3.5443 | |
| | 3.1431 | 0.4625 | 222000 | 3.5456 | |
| | 3.0537 | 0.4646 | 223000 | 3.5422 | |
| | 3.1322 | 0.4667 | 224000 | 3.5385 | |
| | 3.1526 | 0.4688 | 225000 | 3.5361 | |
| | 3.0659 | 0.4708 | 226000 | 3.5349 | |
| | 3.13 | 0.4729 | 227000 | 3.5297 | |
| | 3.1159 | 0.475 | 228000 | 3.5287 | |
| | 3.0748 | 0.4771 | 229000 | 3.5267 | |
| | 3.1242 | 0.4792 | 230000 | 3.5354 | |
| | 3.1809 | 0.4813 | 231000 | 3.5289 | |
| | 3.1093 | 0.4833 | 232000 | 3.5316 | |
| | 3.1534 | 0.4854 | 233000 | 3.5325 | |
| | 3.134 | 0.4875 | 234000 | 3.5261 | |
| | 3.1468 | 0.4896 | 235000 | 3.5295 | |
| | 3.1688 | 0.4917 | 236000 | 3.5239 | |
| | 3.1643 | 0.4938 | 237000 | 3.5291 | |
| | 3.1373 | 0.4958 | 238000 | 3.5197 | |
| | 3.1588 | 0.4979 | 239000 | 3.5217 | |
| | 3.0473 | 0.5 | 240000 | 3.5267 | |
| | 3.2146 | 0.5021 | 241000 | 3.5212 | |
| | 3.1697 | 0.5042 | 242000 | 3.5290 | |
| | 3.148 | 0.5062 | 243000 | 3.5268 | |
| | 3.0906 | 0.5083 | 244000 | 3.5178 | |
| | 3.1083 | 0.5104 | 245000 | 3.5126 | |
| | 3.1061 | 0.5125 | 246000 | 3.5130 | |
| | 3.111 | 0.5146 | 247000 | 3.4991 | |
| | 3.1511 | 0.5167 | 248000 | 3.5115 | |
| | 3.1264 | 0.5188 | 249000 | 3.5141 | |
| | 3.2034 | 0.5208 | 250000 | 3.5121 | |
| | 3.06 | 0.5229 | 251000 | 3.5153 | |
| | 3.0718 | 0.525 | 252000 | 3.5144 | |
| | 3.0548 | 0.5271 | 253000 | 3.5069 | |
| | 3.1063 | 0.5292 | 254000 | 3.4987 | |
| | 3.0995 | 0.5312 | 255000 | 3.5105 | |
| | 3.1381 | 0.5333 | 256000 | 3.5025 | |
| | 3.0581 | 0.5354 | 257000 | 3.5044 | |
| | 3.1133 | 0.5375 | 258000 | 3.4979 | |
| | 3.1613 | 0.5396 | 259000 | 3.4933 | |
| | 3.0973 | 0.5417 | 260000 | 3.5024 | |
| | 3.0427 | 0.5437 | 261000 | 3.5095 | |
| | 3.0705 | 0.5458 | 262000 | 3.5020 | |
| | 3.1001 | 0.5479 | 263000 | 3.5021 | |
| | 3.0793 | 0.55 | 264000 | 3.5032 | |
| | 3.1197 | 0.5521 | 265000 | 3.5044 | |
| | 3.0562 | 0.5542 | 266000 | 3.5001 | |
| | 3.1128 | 0.5563 | 267000 | 3.4976 | |
| | 3.073 | 0.5583 | 268000 | 3.5002 | |
| | 3.0071 | 0.5604 | 269000 | 3.4963 | |
| | 3.0537 | 0.5625 | 270000 | 3.5080 | |
| | 3.0515 | 0.5646 | 271000 | 3.5022 | |
| | 2.8561 | 0.5667 | 272000 | 3.4993 | |
| | 2.1328 | 0.5687 | 273000 | 3.5052 | |
| | 1.8604 | 0.5708 | 274000 | 3.5099 | |
| | 1.5633 | 0.5729 | 275000 | 3.5245 | |
| | 1.4802 | 0.575 | 276000 | 3.5352 | |
| | 1.4986 | 0.5771 | 277000 | 3.5362 | |
| | 1.5193 | 0.5792 | 278000 | 3.5498 | |
| | 1.5455 | 0.5813 | 279000 | 3.5489 | |
| | 1.5169 | 0.5833 | 280000 | 3.5541 | |
| | 1.4809 | 0.5854 | 281000 | 3.5662 | |
| | 1.4723 | 0.5875 | 282000 | 3.5740 | |
| | 1.4385 | 0.5896 | 283000 | 3.5717 | |
| | 1.3069 | 0.5917 | 284000 | 3.5847 | |
| | 1.5347 | 0.5938 | 285000 | 3.5832 | |
| | 1.3994 | 0.5958 | 286000 | 3.5954 | |
| | 1.3792 | 0.5979 | 287000 | 3.5923 | |
| | 1.415 | 0.6 | 288000 | 3.6055 | |
| | 1.5194 | 0.6021 | 289000 | 3.6088 | |
| | 1.29 | 0.6042 | 290000 | 3.6132 | |
| | 1.3464 | 0.6062 | 291000 | 3.6106 | |
| | 1.353 | 0.6083 | 292000 | 3.6079 | |
| | 1.3774 | 0.6104 | 293000 | 3.6108 | |
| | 1.4167 | 0.6125 | 294000 | 3.6117 | |
| | 1.2833 | 0.6146 | 295000 | 3.6048 | |
| | 1.344 | 0.6167 | 296000 | 3.6066 | |
| | 1.4293 | 0.6188 | 297000 | 3.6185 | |
| | 1.2332 | 0.6208 | 298000 | 3.6221 | |
| | 1.2435 | 0.6229 | 299000 | 3.6140 | |
| | 1.2532 | 0.625 | 300000 | 3.6197 | |
| | 1.2261 | 0.6271 | 301000 | 3.6293 | |
| | 1.9938 | 0.6292 | 302000 | 3.5898 | |
| | 2.8128 | 0.6312 | 303000 | 3.5598 | |
| | 2.9367 | 0.6333 | 304000 | 3.5419 | |
| | 3.0334 | 0.6354 | 305000 | 3.5330 | |
| | 3.1232 | 0.6375 | 306000 | 3.5259 | |
| | 3.0749 | 0.6396 | 307000 | 3.5236 | |
| | 3.1853 | 0.6417 | 308000 | 3.5134 | |
| | 3.1386 | 0.6438 | 309000 | 3.5126 | |
| | 3.1555 | 0.6458 | 310000 | 3.5069 | |
| | 3.0662 | 0.6479 | 311000 | 3.5027 | |
| | 3.0098 | 0.65 | 312000 | 3.4989 | |
| | 3.0513 | 0.6521 | 313000 | 3.4942 | |
| | 3.081 | 0.6542 | 314000 | 3.4905 | |
| | 3.0592 | 0.6562 | 315000 | 3.4902 | |
| | 3.0992 | 0.6583 | 316000 | 3.4897 | |
| | 3.0761 | 0.6604 | 317000 | 3.4902 | |
| | 3.0232 | 0.6625 | 318000 | 3.4816 | |
| | 3.0426 | 0.6646 | 319000 | 3.4819 | |
| | 3.1092 | 0.6667 | 320000 | 3.4777 | |
| | 3.0312 | 0.6687 | 321000 | 3.4893 | |
| | 3.0898 | 0.6708 | 322000 | 3.4896 | |
| | 3.0576 | 0.6729 | 323000 | 3.4824 | |
| | 2.9931 | 0.675 | 324000 | 3.4772 | |
| | 3.0077 | 0.6771 | 325000 | 3.4817 | |
| | 3.0831 | 0.6792 | 326000 | 3.4760 | |
| | 3.1157 | 0.6813 | 327000 | 3.4756 | |
| | 3.0822 | 0.6833 | 328000 | 3.4752 | |
| | 3.1263 | 0.6854 | 329000 | 3.4697 | |
| | 3.0483 | 0.6875 | 330000 | 3.4760 | |
| | 3.0984 | 0.6896 | 331000 | 3.4763 | |
| | 3.0595 | 0.6917 | 332000 | 3.4730 | |
| | 3.0526 | 0.6937 | 333000 | 3.4710 | |
| | 3.0795 | 0.6958 | 334000 | 3.4736 | |
| | 3.0524 | 0.6979 | 335000 | 3.4734 | |
| | 3.0192 | 0.7 | 336000 | 3.4656 | |
| | 3.0174 | 0.7021 | 337000 | 3.4687 | |
| | 3.0312 | 0.7042 | 338000 | 3.4715 | |
| | 3.0186 | 0.7063 | 339000 | 3.4701 | |
| | 3.0222 | 0.7083 | 340000 | 3.4639 | |
| | 3.0076 | 0.7104 | 341000 | 3.4650 | |
| | 3.0714 | 0.7125 | 342000 | 3.4623 | |
| | 2.9175 | 0.7146 | 343000 | 3.4622 | |
| | 3.0136 | 0.7167 | 344000 | 3.4644 | |
| | 3.0539 | 0.7188 | 345000 | 3.4577 | |
| | 3.0864 | 0.7208 | 346000 | 3.4561 | |
| | 3.0607 | 0.7229 | 347000 | 3.4545 | |
| | 3.0489 | 0.725 | 348000 | 3.4531 | |
| | 3.0048 | 0.7271 | 349000 | 3.4567 | |
| | 3.0989 | 0.7292 | 350000 | 3.4562 | |
| | 3.0388 | 0.7312 | 351000 | 3.4571 | |
| | 3.0766 | 0.7333 | 352000 | 3.4533 | |
| | 3.084 | 0.7354 | 353000 | 3.4575 | |
| | 3.0652 | 0.7375 | 354000 | 3.4512 | |
| | 3.0437 | 0.7396 | 355000 | 3.4551 | |
| | 3.0438 | 0.7417 | 356000 | 3.4570 | |
| | 3.081 | 0.7438 | 357000 | 3.4536 | |
| | 3.0803 | 0.7458 | 358000 | 3.4533 | |
| | 3.0359 | 0.7479 | 359000 | 3.4584 | |
| | 3.0631 | 0.75 | 360000 | 3.4593 | |
| | 3.0193 | 0.7521 | 361000 | 3.4537 | |
| | 3.1205 | 0.7542 | 362000 | 3.4506 | |
| | 3.0604 | 0.7562 | 363000 | 3.4502 | |
| | 3.0792 | 0.7583 | 364000 | 3.4503 | |
| | 2.9729 | 0.7604 | 365000 | 3.4479 | |
| | 3.0601 | 0.7625 | 366000 | 3.4429 | |
| | 3.0352 | 0.7646 | 367000 | 3.4464 | |
| | 3.0392 | 0.7667 | 368000 | 3.4460 | |
| | 3.066 | 0.7688 | 369000 | 3.4478 | |
| | 3.0074 | 0.7708 | 370000 | 3.4499 | |
| | 3.0857 | 0.7729 | 371000 | 3.4460 | |
| | 3.0815 | 0.775 | 372000 | 3.4425 | |
| | 3.0046 | 0.7771 | 373000 | 3.4449 | |
| | 3.0233 | 0.7792 | 374000 | 3.4455 | |
| | 3.0774 | 0.7812 | 375000 | 3.4469 | |
| | 3.0053 | 0.7833 | 376000 | 3.4447 | |
| | 3.0367 | 0.7854 | 377000 | 3.4424 | |
| | 3.0685 | 0.7875 | 378000 | 3.4449 | |
| | 3.0189 | 0.7896 | 379000 | 3.4436 | |
| | 3.0873 | 0.7917 | 380000 | 3.4360 | |
| | 3.0593 | 0.7937 | 381000 | 3.4390 | |
| | 2.9734 | 0.7958 | 382000 | 3.4363 | |
| | 2.9926 | 0.7979 | 383000 | 3.4385 | |
| | 3.0301 | 0.8 | 384000 | 3.4374 | |
| | 2.9467 | 0.8021 | 385000 | 3.4328 | |
| | 2.9794 | 0.8042 | 386000 | 3.4346 | |
| | 3.0649 | 0.8063 | 387000 | 3.4329 | |
| | 3.0733 | 0.8083 | 388000 | 3.4324 | |
| | 3.0386 | 0.8104 | 389000 | 3.4332 | |
| | 3.013 | 0.8125 | 390000 | 3.4350 | |
| | 3.0386 | 0.8146 | 391000 | 3.4324 | |
| | 3.0885 | 0.8167 | 392000 | 3.4341 | |
| | 2.996 | 0.8187 | 393000 | 3.4339 | |
| | 3.1084 | 0.8208 | 394000 | 3.4343 | |
| | 3.0 | 0.8229 | 395000 | 3.4316 | |
| | 2.9807 | 0.825 | 396000 | 3.4270 | |
| | 3.0788 | 0.8271 | 397000 | 3.4272 | |
| | 3.0419 | 0.8292 | 398000 | 3.4302 | |
| | 2.9872 | 0.8313 | 399000 | 3.4274 | |
| | 2.9853 | 0.8333 | 400000 | 3.4311 | |
| | 3.2761 | 0.8354 | 401000 | 3.4221 | |
| | 3.4203 | 0.8375 | 402000 | 3.4176 | |
| | 3.3776 | 0.8396 | 403000 | 3.4127 | |
| | 3.4266 | 0.8417 | 404000 | 3.4108 | |
| | 3.4513 | 0.8438 | 405000 | 3.4067 | |
| | 3.417 | 0.8458 | 406000 | 3.4066 | |
| | 3.371 | 0.8479 | 407000 | 3.4048 | |
| | 3.4803 | 0.85 | 408000 | 3.4022 | |
| | 3.4137 | 0.8521 | 409000 | 3.4028 | |
| | 3.3304 | 0.8542 | 410000 | 3.4032 | |
| | 3.3579 | 0.8562 | 411000 | 3.4039 | |
| | 3.3827 | 0.8583 | 412000 | 3.3999 | |
| | 3.3244 | 0.8604 | 413000 | 3.3995 | |
| | 3.3754 | 0.8625 | 414000 | 3.3995 | |
| | 3.3883 | 0.8646 | 415000 | 3.3993 | |
| | 3.4301 | 0.8667 | 416000 | 3.3979 | |
| | 3.4519 | 0.8688 | 417000 | 3.3980 | |
| | 3.4338 | 0.8708 | 418000 | 3.3968 | |
| | 3.5211 | 0.8729 | 419000 | 3.3981 | |
| | 3.4089 | 0.875 | 420000 | 3.3970 | |
| | 3.376 | 0.8771 | 421000 | 3.3965 | |
| | 3.3782 | 0.8792 | 422000 | 3.3946 | |
| | 3.4154 | 0.8812 | 423000 | 3.3910 | |
| | 3.3131 | 0.8833 | 424000 | 3.3906 | |
| | 3.3433 | 0.8854 | 425000 | 3.3923 | |
| | 3.4678 | 0.8875 | 426000 | 3.3893 | |
| | 3.3821 | 0.8896 | 427000 | 3.3873 | |
| | 3.4541 | 0.8917 | 428000 | 3.3896 | |
| | 3.3757 | 0.8938 | 429000 | 3.3880 | |
| | 3.4463 | 0.8958 | 430000 | 3.3897 | |
| | 3.4266 | 0.8979 | 431000 | 3.3864 | |
| | 3.3054 | 0.9 | 432000 | 3.3854 | |
| | 3.4259 | 0.9021 | 433000 | 3.3856 | |
| | 3.3983 | 0.9042 | 434000 | 3.3869 | |
| | 3.3211 | 0.9062 | 435000 | 3.3866 | |
| | 3.416 | 0.9083 | 436000 | 3.3850 | |
| | 3.3945 | 0.9104 | 437000 | 3.3847 | |
| | 3.4763 | 0.9125 | 438000 | 3.3849 | |
| | 3.3599 | 0.9146 | 439000 | 3.3836 | |
| | 3.4208 | 0.9167 | 440000 | 3.3843 | |
| | 3.4234 | 0.9187 | 441000 | 3.3829 | |
| | 3.3527 | 0.9208 | 442000 | 3.3838 | |
| | 3.3731 | 0.9229 | 443000 | 3.3830 | |
| | 3.3564 | 0.925 | 444000 | 3.3829 | |
| | 3.4002 | 0.9271 | 445000 | 3.3823 | |
| | 3.4151 | 0.9292 | 446000 | 3.3819 | |
| | 3.3721 | 0.9313 | 447000 | 3.3814 | |
| | 3.3909 | 0.9333 | 448000 | 3.3819 | |
| | 3.3552 | 0.9354 | 449000 | 3.3820 | |
| | 3.3308 | 0.9375 | 450000 | 3.3837 | |
| | 3.3509 | 0.9396 | 451000 | 3.3846 | |
| | 3.404 | 0.9417 | 452000 | 3.3846 | |
| | 3.3481 | 0.9437 | 453000 | 3.3842 | |
| | 3.383 | 0.9458 | 454000 | 3.3844 | |
| | 3.3528 | 0.9479 | 455000 | 3.3834 | |
| | 3.3573 | 0.95 | 456000 | 3.3832 | |
| | 3.3597 | 0.9521 | 457000 | 3.3830 | |
| | 3.3821 | 0.9542 | 458000 | 3.3831 | |
| | 3.4154 | 0.9563 | 459000 | 3.3828 | |
| | 3.3741 | 0.9583 | 460000 | 3.3815 | |
| | 3.4097 | 0.9604 | 461000 | 3.3813 | |
| | 3.5173 | 0.9625 | 462000 | 3.3824 | |
| | 3.3884 | 0.9646 | 463000 | 3.3817 | |
| | 3.3439 | 0.9667 | 464000 | 3.3814 | |
| | 3.3791 | 0.9688 | 465000 | 3.3814 | |
| | 3.4821 | 0.9708 | 466000 | 3.3806 | |
| | 3.3939 | 0.9729 | 467000 | 3.3808 | |
| | 3.426 | 0.975 | 468000 | 3.3814 | |
| | 3.3527 | 0.9771 | 469000 | 3.3811 | |
| | 3.3875 | 0.9792 | 470000 | 3.3811 | |
| | 3.3621 | 0.9812 | 471000 | 3.3810 | |
| | 3.406 | 0.9833 | 472000 | 3.3807 | |
| | 3.4422 | 0.9854 | 473000 | 3.3813 | |
| | 3.465 | 0.9875 | 474000 | 3.3815 | |
| | 3.3671 | 0.9896 | 475000 | 3.3809 | |
| | 3.4621 | 0.9917 | 476000 | 3.3815 | |
| | 3.4379 | 0.9938 | 477000 | 3.3810 | |
| | 3.3686 | 0.9958 | 478000 | 3.3813 | |
| | 3.4255 | 0.9979 | 479000 | 3.3811 | |
| | 3.3782 | 1.0 | 480000 | 3.3814 | |
|
|
|
|
| ### Framework versions |
|
|
| - Transformers 4.50.3 |
| - Pytorch 2.6.0+cu124 |
| - Datasets 2.21.0 |
| - Tokenizers 0.21.4 |
|
|
