--- library_name: transformers license: other base_model: google/gemma-3-270m tags: - llama-factory - full - generated_from_trainer model-index: - name: gemma3-270m-pt results: [] --- # gemma3-270m-pt This model is a fine-tuned version of [google/gemma-3-270m](https://huggingface.co/google/gemma-3-270m) on the dolma_v17 dataset. It achieves the following results on the evaluation set: - Loss: 3.5198 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0003 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 2000 - training_steps: 480000 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:------:|:---------------:| | 4.2931 | 0.0021 | 1000 | 3.9822 | | 4.6323 | 0.0042 | 2000 | 4.4674 | | 4.745 | 0.0063 | 3000 | 4.4693 | | 4.8921 | 0.0083 | 4000 | 4.3719 | | 4.8514 | 0.0104 | 5000 | 4.3759 | | 4.4932 | 0.0125 | 6000 | 4.3433 | | 4.3918 | 0.0146 | 7000 | 4.2911 | | 4.6039 | 0.0167 | 8000 | 4.2582 | | 4.5943 | 0.0187 | 9000 | 4.2454 | | 4.3709 | 0.0208 | 10000 | 4.1994 | | 4.1869 | 0.0229 | 11000 | 4.1839 | | 4.4256 | 0.025 | 12000 | 4.1770 | | 4.5704 | 0.0271 | 13000 | 4.1493 | | 4.5747 | 0.0292 | 14000 | 4.1570 | | 4.3402 | 0.0312 | 15000 | 4.1393 | | 4.1566 | 0.0333 | 16000 | 4.1090 | | 4.3037 | 0.0354 | 17000 | 4.1317 | | 4.2857 | 0.0375 | 18000 | 4.1093 | | 4.2942 | 0.0396 | 19000 | 4.0916 | | 4.5122 | 0.0417 | 20000 | 4.0791 | | 4.1701 | 0.0437 | 21000 | 4.0935 | | 4.3904 | 0.0458 | 22000 | 4.0831 | | 4.2763 | 0.0479 | 23000 | 4.0704 | | 4.2784 | 0.05 | 24000 | 4.0572 | | 4.2718 | 0.0521 | 25000 | 4.0627 | | 4.3324 | 0.0542 | 26000 | 4.0632 | | 4.2779 | 0.0563 | 27000 | 4.0544 | | 4.3344 | 0.0583 | 28000 | 4.0399 | | 4.322 | 0.0604 | 29000 | 4.0418 | | 4.1958 | 0.0625 | 30000 | 4.0252 | | 4.3026 | 0.0646 | 31000 | 4.0048 | | 4.2979 | 0.0667 | 32000 | 3.9925 | | 4.305 | 0.0688 | 33000 | 3.9766 | | 4.3174 | 0.0708 | 34000 | 3.9766 | | 4.2148 | 0.0729 | 35000 | 3.9660 | | 4.2765 | 0.075 | 36000 | 3.9570 | | 4.2918 | 0.0771 | 37000 | 3.9527 | | 4.3365 | 0.0792 | 38000 | 3.9565 | | 4.4254 | 0.0813 | 39000 | 3.9651 | | 4.1312 | 0.0833 | 40000 | 3.9580 | | 4.3529 | 0.0854 | 41000 | 3.9465 | | 4.275 | 0.0875 | 42000 | 3.9280 | | 4.1126 | 0.0896 | 43000 | 3.9068 | | 4.2748 | 0.0917 | 44000 | 3.9242 | | 4.3029 | 0.0938 | 45000 | 3.9233 | | 4.2411 | 0.0958 | 46000 | 3.9290 | | 4.1393 | 0.0979 | 47000 | 3.9316 | | 4.2889 | 0.1 | 48000 | 3.9284 | | 4.0885 | 0.1021 | 49000 | 3.9186 | | 4.1743 | 0.1042 | 50000 | 3.9073 | | 4.255 | 0.1062 | 51000 | 3.9065 | | 4.2229 | 0.1083 | 52000 | 3.9164 | | 4.191 | 0.1104 | 53000 | 3.9075 | | 4.0171 | 0.1125 | 54000 | 3.9135 | | 4.1305 | 0.1146 | 55000 | 3.9226 | | 4.1116 | 0.1167 | 56000 | 3.9066 | | 4.1122 | 0.1187 | 57000 | 3.9025 | | 4.2754 | 0.1208 | 58000 | 3.8955 | | 4.0923 | 0.1229 | 59000 | 3.8926 | | 3.9204 | 0.125 | 60000 | 3.8967 | | 4.048 | 0.1271 | 61000 | 3.8833 | | 4.17 | 0.1292 | 62000 | 3.8793 | | 4.0509 | 0.1313 | 63000 | 3.8943 | | 3.9868 | 0.1333 | 64000 | 3.8808 | | 4.1509 | 0.1354 | 65000 | 3.8838 | | 4.0643 | 0.1375 | 66000 | 3.8647 | | 4.3223 | 0.1396 | 67000 | 3.8736 | | 4.1562 | 0.1417 | 68000 | 3.8756 | | 4.0332 | 0.1437 | 69000 | 3.8688 | | 4.2808 | 0.1458 | 70000 | 3.8694 | | 4.1921 | 0.1479 | 71000 | 3.8743 | | 4.2636 | 0.15 | 72000 | 3.8673 | | 4.1798 | 0.1521 | 73000 | 3.8729 | | 4.1839 | 0.1542 | 74000 | 3.8714 | | 4.1718 | 0.1562 | 75000 | 3.8536 | | 4.0316 | 0.1583 | 76000 | 3.8605 | | 4.01 | 0.1604 | 77000 | 3.8588 | | 4.0895 | 0.1625 | 78000 | 3.8590 | | 4.1559 | 0.1646 | 79000 | 3.8427 | | 3.9592 | 0.1667 | 80000 | 3.8471 | | 4.0736 | 0.1688 | 81000 | 3.8464 | | 4.0589 | 0.1708 | 82000 | 3.8426 | | 4.0019 | 0.1729 | 83000 | 3.8348 | | 4.1604 | 0.175 | 84000 | 3.8170 | | 4.0986 | 0.1771 | 85000 | 3.8415 | | 4.07 | 0.1792 | 86000 | 3.8292 | | 4.0258 | 0.1812 | 87000 | 3.8090 | | 3.774 | 0.1833 | 88000 | 3.8328 | | 3.994 | 0.1854 | 89000 | 3.8348 | | 4.1861 | 0.1875 | 90000 | 3.8286 | | 4.0522 | 0.1896 | 91000 | 3.8463 | | 4.0582 | 0.1917 | 92000 | 3.8335 | | 4.2788 | 0.1938 | 93000 | 3.8240 | | 4.1649 | 0.1958 | 94000 | 3.8365 | | 3.9257 | 0.1979 | 95000 | 3.8402 | | 4.0108 | 0.2 | 96000 | 3.8102 | | 4.0671 | 0.2021 | 97000 | 3.8115 | | 3.9055 | 0.2042 | 98000 | 3.8124 | | 4.148 | 0.2062 | 99000 | 3.8110 | | 4.1905 | 0.2083 | 100000 | 3.8097 | | 4.0249 | 0.2104 | 101000 | 3.8164 | | 4.0572 | 0.2125 | 102000 | 3.8098 | | 4.0048 | 0.2146 | 103000 | 3.7977 | | 4.0949 | 0.2167 | 104000 | 3.7843 | | 4.0705 | 0.2188 | 105000 | 3.7880 | | 3.9206 | 0.2208 | 106000 | 3.7837 | | 4.177 | 0.2229 | 107000 | 3.7837 | | 4.0109 | 0.225 | 108000 | 3.7857 | | 4.0119 | 0.2271 | 109000 | 3.7832 | | 4.0331 | 0.2292 | 110000 | 3.7930 | | 4.1279 | 0.2313 | 111000 | 3.7909 | | 4.1385 | 0.2333 | 112000 | 3.7726 | | 3.9555 | 0.2354 | 113000 | 3.7787 | | 4.0349 | 0.2375 | 114000 | 3.7686 | | 3.9498 | 0.2396 | 115000 | 3.7734 | | 4.0348 | 0.2417 | 116000 | 3.7677 | | 4.1393 | 0.2437 | 117000 | 3.7777 | | 3.9934 | 0.2458 | 118000 | 3.7830 | | 4.1066 | 0.2479 | 119000 | 3.7687 | | 3.9003 | 0.25 | 120000 | 3.7754 | | 3.9331 | 0.2521 | 121000 | 3.7839 | | 3.9418 | 0.2542 | 122000 | 3.7685 | | 3.9204 | 0.2562 | 123000 | 3.7759 | | 4.0853 | 0.2583 | 124000 | 3.7626 | | 4.0488 | 0.2604 | 125000 | 3.7654 | | 4.0562 | 0.2625 | 126000 | 3.7541 | | 4.0137 | 0.2646 | 127000 | 3.7602 | | 4.0882 | 0.2667 | 128000 | 3.7574 | | 4.0871 | 0.2687 | 129000 | 3.7381 | | 4.0727 | 0.2708 | 130000 | 3.7511 | | 3.9283 | 0.2729 | 131000 | 3.7516 | | 4.2202 | 0.275 | 132000 | 3.7495 | | 4.1319 | 0.2771 | 133000 | 3.7491 | | 4.3493 | 0.2792 | 134000 | 3.7484 | | 3.9107 | 0.2812 | 135000 | 3.7510 | | 3.9319 | 0.2833 | 136000 | 3.7488 | | 4.1152 | 0.2854 | 137000 | 3.7645 | | 3.9751 | 0.2875 | 138000 | 3.7466 | | 3.8191 | 0.2896 | 139000 | 3.7461 | | 4.0677 | 0.2917 | 140000 | 3.7349 | | 4.1179 | 0.2938 | 141000 | 3.7402 | | 3.8622 | 0.2958 | 142000 | 3.7305 | | 3.7921 | 0.2979 | 143000 | 3.7411 | | 4.1718 | 0.3 | 144000 | 3.7449 | | 4.0752 | 0.3021 | 145000 | 3.7485 | | 3.9686 | 0.3042 | 146000 | 3.7388 | | 4.3016 | 0.3063 | 147000 | 3.7321 | | 4.1476 | 0.3083 | 148000 | 3.7399 | | 3.9213 | 0.3104 | 149000 | 3.7481 | | 3.9286 | 0.3125 | 150000 | 3.7312 | | 4.0123 | 0.3146 | 151000 | 3.7343 | | 3.9552 | 0.3167 | 152000 | 3.7323 | | 4.0371 | 0.3187 | 153000 | 3.7402 | | 4.121 | 0.3208 | 154000 | 3.7408 | | 3.7645 | 0.3229 | 155000 | 3.7556 | | 3.8525 | 0.325 | 156000 | 3.7431 | | 3.9261 | 0.3271 | 157000 | 3.7377 | | 3.9016 | 0.3292 | 158000 | 3.7360 | | 3.7874 | 0.3312 | 159000 | 3.7338 | | 3.8374 | 0.3333 | 160000 | 3.7305 | | 3.9809 | 0.3354 | 161000 | 3.7291 | | 3.9749 | 0.3375 | 162000 | 3.7477 | | 4.027 | 0.3396 | 163000 | 3.7321 | | 3.9302 | 0.3417 | 164000 | 3.7238 | | 3.9177 | 0.3438 | 165000 | 3.7137 | | 4.0352 | 0.3458 | 166000 | 3.7024 | | 4.0522 | 0.3479 | 167000 | 3.7127 | | 3.7986 | 0.35 | 168000 | 3.7101 | | 3.8614 | 0.3521 | 169000 | 3.7270 | | 3.7471 | 0.3542 | 170000 | 3.7230 | | 3.99 | 0.3563 | 171000 | 3.7103 | | 3.9798 | 0.3583 | 172000 | 3.6977 | | 3.9396 | 0.3604 | 173000 | 3.6922 | | 3.9882 | 0.3625 | 174000 | 3.6892 | | 3.9534 | 0.3646 | 175000 | 3.6867 | | 3.9774 | 0.3667 | 176000 | 3.6919 | | 3.8443 | 0.3688 | 177000 | 3.6836 | | 3.8834 | 0.3708 | 178000 | 3.6745 | | 3.7871 | 0.3729 | 179000 | 3.6850 | | 4.0395 | 0.375 | 180000 | 3.6802 | | 3.9295 | 0.3771 | 181000 | 3.6699 | | 3.9957 | 0.3792 | 182000 | 3.6829 | | 4.0249 | 0.3812 | 183000 | 3.6728 | | 3.7347 | 0.3833 | 184000 | 3.6613 | | 3.9043 | 0.3854 | 185000 | 3.6630 | | 3.9551 | 0.3875 | 186000 | 3.6611 | | 3.906 | 0.3896 | 187000 | 3.6649 | | 4.3365 | 0.3917 | 188000 | 3.6757 | | 4.0758 | 0.3937 | 189000 | 3.6666 | | 3.9797 | 0.3958 | 190000 | 3.6708 | | 3.9825 | 0.3979 | 191000 | 3.6633 | | 3.8849 | 0.4 | 192000 | 3.6632 | | 3.9139 | 0.4021 | 193000 | 3.6832 | | 3.9537 | 0.4042 | 194000 | 3.6711 | | 4.0484 | 0.4062 | 195000 | 3.6686 | | 4.1876 | 0.4083 | 196000 | 3.6613 | | 3.8799 | 0.4104 | 197000 | 3.6589 | | 3.7906 | 0.4125 | 198000 | 3.6662 | | 4.0187 | 0.4146 | 199000 | 3.6712 | | 3.8207 | 0.4167 | 200000 | 3.6694 | | 3.7844 | 0.4188 | 201000 | 3.6688 | | 3.8222 | 0.4208 | 202000 | 3.6819 | | 4.0005 | 0.4229 | 203000 | 3.6647 | | 3.7722 | 0.425 | 204000 | 3.6624 | | 3.8445 | 0.4271 | 205000 | 3.6629 | | 3.7918 | 0.4292 | 206000 | 3.6577 | | 3.7915 | 0.4313 | 207000 | 3.6559 | | 3.9268 | 0.4333 | 208000 | 3.6537 | | 3.8381 | 0.4354 | 209000 | 3.6549 | | 4.0074 | 0.4375 | 210000 | 3.6543 | | 3.8656 | 0.4396 | 211000 | 3.6558 | | 4.0065 | 0.4417 | 212000 | 3.6424 | | 3.8932 | 0.4437 | 213000 | 3.6373 | | 3.8183 | 0.4458 | 214000 | 3.6408 | | 3.9002 | 0.4479 | 215000 | 3.6392 | | 4.0546 | 0.45 | 216000 | 3.6456 | | 3.7364 | 0.4521 | 217000 | 3.6402 | | 3.8016 | 0.4542 | 218000 | 3.6185 | | 3.8533 | 0.4562 | 219000 | 3.6372 | | 3.8751 | 0.4583 | 220000 | 3.6174 | | 4.0225 | 0.4604 | 221000 | 3.6358 | | 3.8398 | 0.4625 | 222000 | 3.6328 | | 3.7861 | 0.4646 | 223000 | 3.6325 | | 3.8279 | 0.4667 | 224000 | 3.6420 | | 3.8024 | 0.4688 | 225000 | 3.6426 | | 3.9001 | 0.4708 | 226000 | 3.6377 | | 3.6935 | 0.4729 | 227000 | 3.6371 | | 3.9794 | 0.475 | 228000 | 3.6331 | | 4.0112 | 0.4771 | 229000 | 3.6207 | | 3.8076 | 0.4792 | 230000 | 3.6280 | | 3.9526 | 0.4813 | 231000 | 3.6260 | | 3.9038 | 0.4833 | 232000 | 3.6226 | | 4.0885 | 0.4854 | 233000 | 3.6142 | | 3.7187 | 0.4875 | 234000 | 3.6149 | | 3.7443 | 0.4896 | 235000 | 3.6261 | | 3.8012 | 0.4917 | 236000 | 3.6279 | | 3.8198 | 0.4938 | 237000 | 3.6302 | | 3.8801 | 0.4958 | 238000 | 3.6150 | | 3.9145 | 0.4979 | 239000 | 3.6180 | | 4.0335 | 0.5 | 240000 | 3.6114 | | 3.7209 | 0.5021 | 241000 | 3.6128 | | 3.8876 | 0.5042 | 242000 | 3.6219 | | 4.0172 | 0.5062 | 243000 | 3.6165 | | 3.8156 | 0.5083 | 244000 | 3.6195 | | 3.8998 | 0.5104 | 245000 | 3.6174 | | 4.1189 | 0.5125 | 246000 | 3.6089 | | 3.8895 | 0.5146 | 247000 | 3.6222 | | 3.8921 | 0.5167 | 248000 | 3.6207 | | 3.5137 | 0.5188 | 249000 | 3.6146 | | 3.7088 | 0.5208 | 250000 | 3.6076 | | 4.0513 | 0.5229 | 251000 | 3.6078 | | 3.6733 | 0.525 | 252000 | 3.6110 | | 3.8471 | 0.5271 | 253000 | 3.6098 | | 3.9953 | 0.5292 | 254000 | 3.6035 | | 3.8516 | 0.5312 | 255000 | 3.6044 | | 3.8573 | 0.5333 | 256000 | 3.6095 | | 3.8657 | 0.5354 | 257000 | 3.6039 | | 3.9319 | 0.5375 | 258000 | 3.6050 | | 3.7942 | 0.5396 | 259000 | 3.6041 | | 3.7002 | 0.5417 | 260000 | 3.5967 | | 3.8399 | 0.5437 | 261000 | 3.6015 | | 4.0148 | 0.5458 | 262000 | 3.6024 | | 3.7879 | 0.5479 | 263000 | 3.5933 | | 3.634 | 0.55 | 264000 | 3.5924 | | 3.9587 | 0.5521 | 265000 | 3.6041 | | 3.958 | 0.5542 | 266000 | 3.5933 | | 3.5881 | 0.5563 | 267000 | 3.5859 | | 3.8144 | 0.5583 | 268000 | 3.5920 | | 3.7734 | 0.5604 | 269000 | 3.5971 | | 3.9342 | 0.5625 | 270000 | 3.5972 | | 4.0149 | 0.5646 | 271000 | 3.6040 | | 3.9247 | 0.5667 | 272000 | 3.6030 | | 3.9302 | 0.5687 | 273000 | 3.6036 | | 4.0202 | 0.5708 | 274000 | 3.6041 | | 3.9032 | 0.5729 | 275000 | 3.6014 | | 3.9836 | 0.575 | 276000 | 3.6045 | | 3.8229 | 0.5771 | 277000 | 3.6011 | | 3.8266 | 0.5792 | 278000 | 3.5954 | | 3.8088 | 0.5813 | 279000 | 3.5900 | | 3.8959 | 0.5833 | 280000 | 3.5922 | | 3.8879 | 0.5854 | 281000 | 3.5916 | | 3.8912 | 0.5875 | 282000 | 3.5833 | | 3.9302 | 0.5896 | 283000 | 3.5850 | | 3.8181 | 0.5917 | 284000 | 3.5812 | | 3.9781 | 0.5938 | 285000 | 3.5869 | | 3.8864 | 0.5958 | 286000 | 3.5859 | | 3.9975 | 0.5979 | 287000 | 3.5814 | | 4.0338 | 0.6 | 288000 | 3.5876 | | 3.5784 | 0.6021 | 289000 | 3.5818 | | 3.9342 | 0.6042 | 290000 | 3.5878 | | 3.9703 | 0.6062 | 291000 | 3.5859 | | 3.7855 | 0.6083 | 292000 | 3.5877 | | 3.9546 | 0.6104 | 293000 | 3.5887 | | 3.7679 | 0.6125 | 294000 | 3.5775 | | 3.787 | 0.6146 | 295000 | 3.5849 | | 3.8935 | 0.6167 | 296000 | 3.5790 | | 4.0374 | 0.6188 | 297000 | 3.5771 | | 3.962 | 0.6208 | 298000 | 3.5755 | | 3.9529 | 0.6229 | 299000 | 3.5698 | | 3.7928 | 0.625 | 300000 | 3.5735 | | 3.9283 | 0.6271 | 301000 | 3.5808 | | 3.9167 | 0.6292 | 302000 | 3.5752 | | 4.0339 | 0.6312 | 303000 | 3.5769 | | 3.9226 | 0.6333 | 304000 | 3.5755 | | 3.8948 | 0.6354 | 305000 | 3.5735 | | 4.0062 | 0.6375 | 306000 | 3.5733 | | 3.6993 | 0.6396 | 307000 | 3.5791 | | 3.9134 | 0.6417 | 308000 | 3.5731 | | 3.7685 | 0.6438 | 309000 | 3.5659 | | 3.8434 | 0.6458 | 310000 | 3.5667 | | 3.9332 | 0.6479 | 311000 | 3.5639 | | 3.9069 | 0.65 | 312000 | 3.5631 | | 3.7402 | 0.6521 | 313000 | 3.5590 | | 3.8388 | 0.6542 | 314000 | 3.5600 | | 3.8083 | 0.6562 | 315000 | 3.5654 | | 3.8502 | 0.6583 | 316000 | 3.5590 | | 3.8549 | 0.6604 | 317000 | 3.5587 | | 3.7756 | 0.6625 | 318000 | 3.5482 | | 3.8913 | 0.6646 | 319000 | 3.5507 | | 3.955 | 0.6667 | 320000 | 3.5538 | | 3.8394 | 0.6687 | 321000 | 3.5622 | | 3.8598 | 0.6708 | 322000 | 3.5570 | | 3.5586 | 0.6729 | 323000 | 3.5577 | | 3.924 | 0.675 | 324000 | 3.5536 | | 3.6527 | 0.6771 | 325000 | 3.5447 | | 3.4882 | 0.6792 | 326000 | 3.5492 | | 3.5716 | 0.6813 | 327000 | 3.5459 | | 3.5226 | 0.6833 | 328000 | 3.5486 | | 3.6025 | 0.6854 | 329000 | 3.5483 | | 3.5451 | 0.6875 | 330000 | 3.5502 | | 3.5571 | 0.6896 | 331000 | 3.5465 | | 3.3876 | 0.6917 | 332000 | 3.5445 | | 3.5703 | 0.6937 | 333000 | 3.5495 | | 3.4847 | 0.6958 | 334000 | 3.5489 | | 3.4541 | 0.6979 | 335000 | 3.5533 | | 3.4819 | 0.7 | 336000 | 3.5477 | | 3.4938 | 0.7021 | 337000 | 3.5479 | | 3.4165 | 0.7042 | 338000 | 3.5454 | | 3.4591 | 0.7063 | 339000 | 3.5417 | | 3.4617 | 0.7083 | 340000 | 3.5424 | | 3.4902 | 0.7104 | 341000 | 3.5453 | | 3.444 | 0.7125 | 342000 | 3.5435 | | 3.4484 | 0.7146 | 343000 | 3.5454 | | 3.4559 | 0.7167 | 344000 | 3.5455 | | 3.4761 | 0.7188 | 345000 | 3.5353 | | 3.4726 | 0.7208 | 346000 | 3.5423 | | 3.4529 | 0.7229 | 347000 | 3.5401 | | 3.3701 | 0.725 | 348000 | 3.5383 | | 3.626 | 0.7271 | 349000 | 3.5437 | | 3.4972 | 0.7292 | 350000 | 3.5458 | | 3.4522 | 0.7312 | 351000 | 3.5430 | | 3.5327 | 0.7333 | 352000 | 3.5428 | | 3.3583 | 0.7354 | 353000 | 3.5364 | | 3.451 | 0.7375 | 354000 | 3.5355 | | 3.4928 | 0.7396 | 355000 | 3.5398 | | 3.5929 | 0.7417 | 356000 | 3.5380 | | 3.3514 | 0.7438 | 357000 | 3.5389 | | 3.5727 | 0.7458 | 358000 | 3.5316 | | 3.3779 | 0.7479 | 359000 | 3.5324 | | 3.3743 | 0.75 | 360000 | 3.5313 | | 3.3345 | 0.7521 | 361000 | 3.5304 | | 3.3228 | 0.7542 | 362000 | 3.5348 | | 3.4517 | 0.7562 | 363000 | 3.5276 | | 3.4694 | 0.7583 | 364000 | 3.5315 | | 3.4131 | 0.7604 | 365000 | 3.5318 | | 3.5836 | 0.7625 | 366000 | 3.5350 | | 3.4308 | 0.7646 | 367000 | 3.5350 | | 3.4213 | 0.7667 | 368000 | 3.5392 | | 3.2751 | 0.7688 | 369000 | 3.5325 | | 3.2523 | 0.7708 | 370000 | 3.5302 | | 3.4594 | 0.7729 | 371000 | 3.5330 | | 3.3494 | 0.775 | 372000 | 3.5341 | | 3.7243 | 0.7771 | 373000 | 3.5328 | | 3.3265 | 0.7792 | 374000 | 3.5355 | | 3.3984 | 0.7812 | 375000 | 3.5318 | | 3.5199 | 0.7833 | 376000 | 3.5300 | | 3.5487 | 0.7854 | 377000 | 3.5292 | | 3.4923 | 0.7875 | 378000 | 3.5327 | | 3.4877 | 0.7896 | 379000 | 3.5324 | | 3.4278 | 0.7917 | 380000 | 3.5271 | | 3.73 | 0.7937 | 381000 | 3.5280 | | 3.4414 | 0.7958 | 382000 | 3.5299 | | 3.5196 | 0.7979 | 383000 | 3.5311 | | 3.5377 | 0.8 | 384000 | 3.5305 | | 3.3851 | 0.8021 | 385000 | 3.5291 | | 3.4128 | 0.8042 | 386000 | 3.5309 | | 3.436 | 0.8063 | 387000 | 3.5338 | | 3.5489 | 0.8083 | 388000 | 3.5270 | | 3.4035 | 0.8104 | 389000 | 3.5288 | | 3.6176 | 0.8125 | 390000 | 3.5243 | | 3.4391 | 0.8146 | 391000 | 3.5285 | | 3.5153 | 0.8167 | 392000 | 3.5283 | | 3.4573 | 0.8187 | 393000 | 3.5286 | | 3.6216 | 0.8208 | 394000 | 3.5250 | | 3.3743 | 0.8229 | 395000 | 3.5280 | | 3.5928 | 0.825 | 396000 | 3.5267 | | 3.5176 | 0.8271 | 397000 | 3.5234 | | 3.5475 | 0.8292 | 398000 | 3.5268 | | 3.4649 | 0.8313 | 399000 | 3.5280 | | 3.3833 | 0.8333 | 400000 | 3.5288 | | 3.3906 | 0.8354 | 401000 | 3.5249 | | 3.3326 | 0.8375 | 402000 | 3.5261 | | 3.3773 | 0.8396 | 403000 | 3.5245 | | 3.3516 | 0.8417 | 404000 | 3.5283 | | 3.5058 | 0.8438 | 405000 | 3.5256 | | 3.426 | 0.8458 | 406000 | 3.5306 | | 3.5362 | 0.8479 | 407000 | 3.5299 | | 3.4152 | 0.85 | 408000 | 3.5257 | | 3.4078 | 0.8521 | 409000 | 3.5295 | | 3.4935 | 0.8542 | 410000 | 3.5277 | | 3.3589 | 0.8562 | 411000 | 3.5302 | | 3.3062 | 0.8583 | 412000 | 3.5257 | | 3.4587 | 0.8604 | 413000 | 3.5269 | | 3.4145 | 0.8625 | 414000 | 3.5246 | | 3.3165 | 0.8646 | 415000 | 3.5241 | | 3.632 | 0.8667 | 416000 | 3.5220 | | 3.3613 | 0.8688 | 417000 | 3.5270 | | 3.3347 | 0.8708 | 418000 | 3.5268 | | 3.3628 | 0.8729 | 419000 | 3.5242 | | 3.455 | 0.875 | 420000 | 3.5285 | | 3.4319 | 0.8771 | 421000 | 3.5281 | | 3.4173 | 0.8792 | 422000 | 3.5292 | | 3.3427 | 0.8812 | 423000 | 3.5273 | | 3.4071 | 0.8833 | 424000 | 3.5264 | | 3.569 | 0.8854 | 425000 | 3.5266 | | 3.4621 | 0.8875 | 426000 | 3.5241 | | 3.4572 | 0.8896 | 427000 | 3.5218 | | 3.5836 | 0.8917 | 428000 | 3.5246 | | 3.4024 | 0.8938 | 429000 | 3.5212 | | 3.3919 | 0.8958 | 430000 | 3.5216 | | 3.4524 | 0.8979 | 431000 | 3.5245 | | 3.3812 | 0.9 | 432000 | 3.5197 | | 3.2617 | 0.9021 | 433000 | 3.5215 | | 3.4425 | 0.9042 | 434000 | 3.5221 | | 3.5493 | 0.9062 | 435000 | 3.5214 | | 3.535 | 0.9083 | 436000 | 3.5223 | | 3.5173 | 0.9104 | 437000 | 3.5231 | | 3.4602 | 0.9125 | 438000 | 3.5203 | | 3.4051 | 0.9146 | 439000 | 3.5236 | | 3.4005 | 0.9167 | 440000 | 3.5224 | | 3.492 | 0.9187 | 441000 | 3.5193 | | 3.4363 | 0.9208 | 442000 | 3.5205 | | 3.5068 | 0.9229 | 443000 | 3.5202 | | 3.4262 | 0.925 | 444000 | 3.5218 | | 3.4604 | 0.9271 | 445000 | 3.5208 | | 3.4551 | 0.9292 | 446000 | 3.5223 | | 3.3616 | 0.9313 | 447000 | 3.5205 | | 3.4318 | 0.9333 | 448000 | 3.5224 | | 3.2792 | 0.9354 | 449000 | 3.5198 | | 3.2764 | 0.9375 | 450000 | 3.5235 | | 3.4373 | 0.9396 | 451000 | 3.5214 | | 3.3766 | 0.9417 | 452000 | 3.5204 | | 3.3295 | 0.9437 | 453000 | 3.5212 | | 3.4599 | 0.9458 | 454000 | 3.5232 | | 3.3857 | 0.9479 | 455000 | 3.5223 | | 3.5271 | 0.95 | 456000 | 3.5235 | | 3.5152 | 0.9521 | 457000 | 3.5208 | | 3.4538 | 0.9542 | 458000 | 3.5203 | | 3.3662 | 0.9563 | 459000 | 3.5190 | | 3.5145 | 0.9583 | 460000 | 3.5200 | | 3.429 | 0.9604 | 461000 | 3.5193 | | 3.4079 | 0.9625 | 462000 | 3.5192 | | 3.4249 | 0.9646 | 463000 | 3.5222 | | 3.2522 | 0.9667 | 464000 | 3.5203 | | 3.392 | 0.9688 | 465000 | 3.5182 | | 3.3134 | 0.9708 | 466000 | 3.5202 | | 3.4052 | 0.9729 | 467000 | 3.5208 | | 3.5185 | 0.975 | 468000 | 3.5201 | | 3.3033 | 0.9771 | 469000 | 3.5206 | | 3.3354 | 0.9792 | 470000 | 3.5187 | | 3.271 | 0.9812 | 471000 | 3.5208 | | 3.4455 | 0.9833 | 472000 | 3.5192 | | 3.4137 | 0.9854 | 473000 | 3.5205 | | 3.3195 | 0.9875 | 474000 | 3.5221 | | 3.3774 | 0.9896 | 475000 | 3.5211 | | 3.412 | 0.9917 | 476000 | 3.5208 | | 3.4131 | 0.9938 | 477000 | 3.5204 | | 3.4231 | 0.9958 | 478000 | 3.5190 | | 3.399 | 0.9979 | 479000 | 3.5210 | | 3.4902 | 1.0 | 480000 | 3.5198 | ### Framework versions - Transformers 4.50.3 - Pytorch 2.6.0+cu124 - Datasets 2.21.0 - Tokenizers 0.21.4