Device: cuda
Loading tokenizer: /tmp/eval/multilingual_32k.model
Loading base model: /tmp/eval/best_model.pt
Model loaded: 3.04B parameters
Loading SFT data from: /tmp/sft_data_v2
Train: 3949348 tokens, Val: 201020 tokens
Using 8-bit AdamW (bitsandbytes)

Starting SFT training for 4000 steps...
Batch size: 1 x 4 accum = 4 effective, Seq len: 2048, LR: 2e-05
Step 10/4000 | Loss: 2.3791 | LR: 0.000001 | TPS: 1196 | 68s
Step 20/4000 | Loss: 2.5346 | LR: 0.000002 | TPS: 1418 | 116s
Step 30/4000 | Loss: 2.7910 | LR: 0.000003 | TPS: 1511 | 163s
Step 40/4000 | Loss: 2.5189 | LR: 0.000004 | TPS: 1562 | 210s
Step 50/4000 | Loss: 2.5049 | LR: 0.000005 | TPS: 1594 | 257s
Step 60/4000 | Loss: 2.5417 | LR: 0.000006 | TPS: 1616 | 304s
Step 70/4000 | Loss: 2.2374 | LR: 0.000007 | TPS: 1633 | 351s
Step 80/4000 | Loss: 2.5328 | LR: 0.000008 | TPS: 1645 | 398s
Step 90/4000 | Loss: 2.5359 | LR: 0.000009 | TPS: 1655 | 445s
Step 100/4000 | Loss: 2.4830 | LR: 0.000010 | TPS: 1663 | 493s
Step 110/4000 | Loss: 2.3015 | LR: 0.000011 | TPS: 1669 | 540s
Step 120/4000 | Loss: 2.4667 | LR: 0.000012 | TPS: 1675 | 587s
Step 130/4000 | Loss: 2.3792 | LR: 0.000013 | TPS: 1680 | 634s
Step 140/4000 | Loss: 2.3918 | LR: 0.000014 | TPS: 1684 | 681s
Step 150/4000 | Loss: 2.3368 | LR: 0.000015 | TPS: 1687 | 728s
Step 160/4000 | Loss: 2.4838 | LR: 0.000016 | TPS: 1690 | 775s
Step 170/4000 | Loss: 2.3578 | LR: 0.000017 | TPS: 1693 | 823s
Step 180/4000 | Loss: 2.5485 | LR: 0.000018 | TPS: 1695 | 870s
Step 190/4000 | Loss: 2.0834 | LR: 0.000019 | TPS: 1698 | 917s
Step 200/4000 | Loss: 1.9784 | LR: 0.000020 | TPS: 1699 | 964s
Step 210/4000 | Loss: 2.4826 | LR: 0.000020 | TPS: 1701 | 1011s
Step 220/4000 | Loss: 2.3540 | LR: 0.000020 | TPS: 1703 | 1058s
Step 230/4000 | Loss: 2.2093 | LR: 0.000020 | TPS: 1704 | 1105s
Step 240/4000 | Loss: 2.2137 | LR: 0.000020 | TPS: 1706 | 1153s
Step 250/4000 | Loss: 2.2151 | LR: 0.000020 | TPS: 1707 | 1200s
Step 260/4000 | Loss: 2.2535 | LR: 0.000020 | TPS: 1708 | 1247s
Step 270/4000 | Loss: 2.2235 | LR: 0.000020 | TPS: 1709 | 1294s
Step 280/4000 | Loss: 2.0449 | LR: 0.000020 | TPS: 1710 | 1341s
Step 290/4000 | Loss: 2.1502 | LR: 0.000020 | TPS: 1711 | 1388s
Step 300/4000 | Loss: 2.3716 | LR: 0.000020 | TPS: 1712 | 1435s
Step 310/4000 | Loss: 2.1591 | LR: 0.000020 | TPS: 1713 | 1483s
Step 320/4000 | Loss: 2.2153 | LR: 0.000020 | TPS: 1714 | 1530s
Step 330/4000 | Loss: 2.2023 | LR: 0.000020 | TPS: 1714 | 1577s
Step 340/4000 | Loss: 2.3968 | LR: 0.000020 | TPS: 1715 | 1624s
Step 350/4000 | Loss: 2.1146 | LR: 0.000020 | TPS: 1716 | 1671s
Step 360/4000 | Loss: 2.1857 | LR: 0.000020 | TPS: 1716 | 1718s
Step 370/4000 | Loss: 2.1965 | LR: 0.000020 | TPS: 1717 | 1765s
Step 380/4000 | Loss: 2.1613 | LR: 0.000020 | TPS: 1717 | 1813s
Step 390/4000 | Loss: 2.3080 | LR: 0.000020 | TPS: 1718 | 1860s
Step 400/4000 | Loss: 2.2964 | LR: 0.000020 | TPS: 1718 | 1907s
  📊 Val loss: 2.2256 (NEW BEST!)
  💾 Best model saved to /tmp/sft/sft_model_v2.pt
Step 410/4000 | Loss: 2.2859 | LR: 0.000020 | TPS: 1703 | 1973s
Step 420/4000 | Loss: 2.1711 | LR: 0.000020 | TPS: 1703 | 2020s
Step 430/4000 | Loss: 2.1434 | LR: 0.000020 | TPS: 1704 | 2067s
Step 440/4000 | Loss: 2.2115 | LR: 0.000020 | TPS: 1705 | 2114s
Step 450/4000 | Loss: 2.2985 | LR: 0.000020 | TPS: 1706 | 2161s
Step 460/4000 | Loss: 1.9845 | LR: 0.000020 | TPS: 1707 | 2208s
Step 470/4000 | Loss: 2.3135 | LR: 0.000020 | TPS: 1707 | 2255s
Step 480/4000 | Loss: 2.3004 | LR: 0.000020 | TPS: 1708 | 2302s
Step 490/4000 | Loss: 2.1841 | LR: 0.000020 | TPS: 1709 | 2349s
Step 500/4000 | Loss: 2.3647 | LR: 0.000020 | TPS: 1709 | 2396s
Step 510/4000 | Loss: 2.1587 | LR: 0.000020 | TPS: 1710 | 2443s
Step 520/4000 | Loss: 2.0790 | LR: 0.000020 | TPS: 1711 | 2490s
Step 530/4000 | Loss: 2.0842 | LR: 0.000020 | TPS: 1711 | 2537s
Step 540/4000 | Loss: 2.4031 | LR: 0.000020 | TPS: 1712 | 2584s
Step 550/4000 | Loss: 2.3037 | LR: 0.000020 | TPS: 1712 | 2632s
Step 560/4000 | Loss: 2.2433 | LR: 0.000020 | TPS: 1713 | 2679s
Step 570/4000 | Loss: 2.1670 | LR: 0.000020 | TPS: 1713 | 2726s
Step 580/4000 | Loss: 2.1579 | LR: 0.000020 | TPS: 1714 | 2773s
Step 590/4000 | Loss: 1.9392 | LR: 0.000020 | TPS: 1714 | 2820s
Step 600/4000 | Loss: 2.1226 | LR: 0.000020 | TPS: 1715 | 2867s
Step 610/4000 | Loss: 2.2641 | LR: 0.000019 | TPS: 1715 | 2914s
Step 620/4000 | Loss: 2.0771 | LR: 0.000019 | TPS: 1715 | 2961s
Step 630/4000 | Loss: 2.4527 | LR: 0.000019 | TPS: 1716 | 3008s
Step 640/4000 | Loss: 2.2605 | LR: 0.000019 | TPS: 1716 | 3055s
Step 650/4000 | Loss: 1.9801 | LR: 0.000019 | TPS: 1717 | 3102s
Step 660/4000 | Loss: 2.4208 | LR: 0.000019 | TPS: 1717 | 3149s
Step 670/4000 | Loss: 2.3331 | LR: 0.000019 | TPS: 1717 | 3196s
Step 680/4000 | Loss: 2.1299 | LR: 0.000019 | TPS: 1718 | 3243s
Step 690/4000 | Loss: 2.1551 | LR: 0.000019 | TPS: 1718 | 3290s
Step 700/4000 | Loss: 2.0940 | LR: 0.000019 | TPS: 1718 | 3337s
Step 710/4000 | Loss: 2.0533 | LR: 0.000019 | TPS: 1719 | 3384s
Step 720/4000 | Loss: 2.2076 | LR: 0.000019 | TPS: 1719 | 3431s
Step 730/4000 | Loss: 1.9816 | LR: 0.000019 | TPS: 1719 | 3478s
Step 740/4000 | Loss: 2.1420 | LR: 0.000019 | TPS: 1719 | 3526s
Step 750/4000 | Loss: 2.2928 | LR: 0.000019 | TPS: 1720 | 3573s
Step 760/4000 | Loss: 2.1035 | LR: 0.000019 | TPS: 1720 | 3620s
Step 770/4000 | Loss: 2.1663 | LR: 0.000019 | TPS: 1720 | 3667s
Step 780/4000 | Loss: 2.2270 | LR: 0.000019 | TPS: 1721 | 3714s
Step 790/4000 | Loss: 2.1436 | LR: 0.000019 | TPS: 1721 | 3761s
Step 800/4000 | Loss: 2.3599 | LR: 0.000019 | TPS: 1721 | 3808s
  📊 Val loss: 2.1960 (NEW BEST!)
  💾 Best model saved to /tmp/sft/sft_model_v2.pt
Step 810/4000 | Loss: 2.2325 | LR: 0.000019 | TPS: 1696 | 3912s
Step 820/4000 | Loss: 2.0798 | LR: 0.000019 | TPS: 1696 | 3960s
Step 830/4000 | Loss: 2.1527 | LR: 0.000019 | TPS: 1697 | 4007s
Step 840/4000 | Loss: 2.2046 | LR: 0.000019 | TPS: 1697 | 4054s
Step 850/4000 | Loss: 2.0648 | LR: 0.000019 | TPS: 1698 | 4101s
Step 860/4000 | Loss: 2.1708 | LR: 0.000019 | TPS: 1698 | 4148s
Step 870/4000 | Loss: 2.3088 | LR: 0.000019 | TPS: 1699 | 4195s
Step 880/4000 | Loss: 1.9936 | LR: 0.000019 | TPS: 1699 | 4242s
Step 890/4000 | Loss: 2.1869 | LR: 0.000019 | TPS: 1700 | 4290s
Step 900/4000 | Loss: 2.4199 | LR: 0.000019 | TPS: 1700 | 4337s
Step 910/4000 | Loss: 2.3803 | LR: 0.000018 | TPS: 1700 | 4384s
Step 920/4000 | Loss: 2.0193 | LR: 0.000018 | TPS: 1701 | 4431s
Step 930/4000 | Loss: 2.1047 | LR: 0.000018 | TPS: 1701 | 4478s
Step 940/4000 | Loss: 2.1449 | LR: 0.000018 | TPS: 1702 | 4525s
Step 950/4000 | Loss: 2.1521 | LR: 0.000018 | TPS: 1702 | 4572s
Step 960/4000 | Loss: 2.2820 | LR: 0.000018 | TPS: 1702 | 4620s
Step 970/4000 | Loss: 2.2996 | LR: 0.000018 | TPS: 1703 | 4667s
Step 980/4000 | Loss: 2.3187 | LR: 0.000018 | TPS: 1703 | 4714s
Step 990/4000 | Loss: 2.1756 | LR: 0.000018 | TPS: 1703 | 4761s
Step 1000/4000 | Loss: 1.9765 | LR: 0.000018 | TPS: 1704 | 4808s

  🔤 Generation samples (step 1000):
    [EN] The capital of France is located in Normandy.
    [HE] מלזיה.
    [AR] باريس.
    [FA] پاریس یکی از شهرهای بزرگ و تاریخی جهان است که دارای جاذبه های طبیعی، فرهنگی و اقتصادی متعددی می باشد. شهر پاریس در غرب کشورمان قرار دارد و به عنوان یکی از مهم ترین مراکز تجاری و مالی دنیا شناخته شده ا
    [TRANSLATE] "תודה על הכול, אבא. אני כאן איתך בכל רגע נתון."

Step 1010/4000 | Loss: 2.1665 | LR: 0.000018 | TPS: 1703 | 4859s
Step 1020/4000 | Loss: 2.1047 | LR: 0.000018 | TPS: 1703 | 4906s
Step 1030/4000 | Loss: 2.2359 | LR: 0.000018 | TPS: 1704 | 4953s
Step 1040/4000 | Loss: 2.0109 | LR: 0.000018 | TPS: 1704 | 5000s
Step 1050/4000 | Loss: 2.1515 | LR: 0.000018 | TPS: 1704 | 5047s
Step 1060/4000 | Loss: 2.0880 | LR: 0.000018 | TPS: 1705 | 5094s
Step 1070/4000 | Loss: 2.2460 | LR: 0.000018 | TPS: 1705 | 5142s
Step 1080/4000 | Loss: 1.9325 | LR: 0.000018 | TPS: 1705 | 5189s
Step 1090/4000 | Loss: 2.2283 | LR: 0.000018 | TPS: 1705 | 5236s
Step 1100/4000 | Loss: 2.3303 | LR: 0.000018 | TPS: 1706 | 5283s
Step 1110/4000 | Loss: 2.1772 | LR: 0.000018 | TPS: 1706 | 5330s
Step 1120/4000 | Loss: 2.1615 | LR: 0.000018 | TPS: 1706 | 5377s
Step 1130/4000 | Loss: 2.1470 | LR: 0.000017 | TPS: 1707 | 5424s
Step 1140/4000 | Loss: 1.9640 | LR: 0.000017 | TPS: 1707 | 5472s
Step 1150/4000 | Loss: 2.1891 | LR: 0.000017 | TPS: 1707 | 5519s
Step 1160/4000 | Loss: 2.2183 | LR: 0.000017 | TPS: 1707 | 5566s
Step 1170/4000 | Loss: 2.0268 | LR: 0.000017 | TPS: 1708 | 5613s
Step 1180/4000 | Loss: 2.2234 | LR: 0.000017 | TPS: 1708 | 5660s
Step 1190/4000 | Loss: 2.1961 | LR: 0.000017 | TPS: 1708 | 5707s
Step 1200/4000 | Loss: 2.2019 | LR: 0.000017 | TPS: 1708 | 5754s
  📊 Val loss: 2.2238 
Step 1210/4000 | Loss: 2.0809 | LR: 0.000017 | TPS: 1707 | 5807s
Step 1220/4000 | Loss: 2.1716 | LR: 0.000017 | TPS: 1707 | 5854s
Step 1230/4000 | Loss: 2.2607 | LR: 0.000017 | TPS: 1707 | 5901s
Step 1240/4000 | Loss: 2.1838 | LR: 0.000017 | TPS: 1708 | 5949s
Step 1250/4000 | Loss: 2.0725 | LR: 0.000017 | TPS: 1708 | 5996s
Step 1260/4000 | Loss: 2.2797 | LR: 0.000017 | TPS: 1708 | 6043s
Step 1270/4000 | Loss: 2.0366 | LR: 0.000017 | TPS: 1708 | 6090s
Step 1280/4000 | Loss: 2.1469 | LR: 0.000017 | TPS: 1709 | 6137s
Step 1290/4000 | Loss: 2.1541 | LR: 0.000017 | TPS: 1709 | 6184s
Step 1300/4000 | Loss: 2.0311 | LR: 0.000017 | TPS: 1709 | 6231s
Step 1310/4000 | Loss: 2.1828 | LR: 0.000016 | TPS: 1709 | 6279s
Step 1320/4000 | Loss: 2.2004 | LR: 0.000016 | TPS: 1709 | 6326s
Step 1330/4000 | Loss: 2.2589 | LR: 0.000016 | TPS: 1710 | 6373s
Step 1340/4000 | Loss: 2.1475 | LR: 0.000016 | TPS: 1710 | 6420s
Step 1350/4000 | Loss: 2.1672 | LR: 0.000016 | TPS: 1710 | 6467s
Step 1360/4000 | Loss: 2.1921 | LR: 0.000016 | TPS: 1710 | 6514s
Step 1370/4000 | Loss: 2.0689 | LR: 0.000016 | TPS: 1710 | 6561s
Step 1380/4000 | Loss: 2.2560 | LR: 0.000016 | TPS: 1711 | 6609s
Step 1390/4000 | Loss: 1.9519 | LR: 0.000016 | TPS: 1711 | 6656s
Step 1400/4000 | Loss: 1.9671 | LR: 0.000016 | TPS: 1711 | 6703s
Step 1410/4000 | Loss: 2.1535 | LR: 0.000016 | TPS: 1711 | 6750s
Step 1420/4000 | Loss: 2.1726 | LR: 0.000016 | TPS: 1711 | 6797s
Step 1430/4000 | Loss: 2.0854 | LR: 0.000016 | TPS: 1712 | 6844s
Step 1440/4000 | Loss: 2.0955 | LR: 0.000016 | TPS: 1712 | 6891s
Step 1450/4000 | Loss: 2.1260 | LR: 0.000016 | TPS: 1712 | 6939s
Step 1460/4000 | Loss: 2.2860 | LR: 0.000016 | TPS: 1712 | 6986s
Step 1470/4000 | Loss: 1.6098 | LR: 0.000015 | TPS: 1712 | 7033s
Step 1480/4000 | Loss: 2.1327 | LR: 0.000015 | TPS: 1712 | 7080s
Step 1490/4000 | Loss: 2.0506 | LR: 0.000015 | TPS: 1713 | 7127s
Step 1500/4000 | Loss: 2.0568 | LR: 0.000015 | TPS: 1713 | 7174s
Step 1510/4000 | Loss: 2.0177 | LR: 0.000015 | TPS: 1713 | 7221s
Step 1520/4000 | Loss: 2.0383 | LR: 0.000015 | TPS: 1713 | 7269s
Step 1530/4000 | Loss: 2.0994 | LR: 0.000015 | TPS: 1713 | 7316s
Step 1540/4000 | Loss: 2.0863 | LR: 0.000015 | TPS: 1713 | 7363s
Step 1550/4000 | Loss: 2.3287 | LR: 0.000015 | TPS: 1714 | 7410s
Step 1560/4000 | Loss: 2.1585 | LR: 0.000015 | TPS: 1714 | 7457s
Step 1570/4000 | Loss: 1.9781 | LR: 0.000015 | TPS: 1714 | 7504s
Step 1580/4000 | Loss: 1.9344 | LR: 0.000015 | TPS: 1714 | 7551s
Step 1590/4000 | Loss: 2.1031 | LR: 0.000015 | TPS: 1714 | 7599s
Step 1600/4000 | Loss: 2.2633 | LR: 0.000015 | TPS: 1714 | 7646s
  📊 Val loss: 2.1164 (NEW BEST!)
  💾 Best model saved to /tmp/sft/sft_model_v2.pt
Step 1610/4000 | Loss: 2.0217 | LR: 0.000015 | TPS: 1702 | 7750s
Step 1620/4000 | Loss: 2.0437 | LR: 0.000014 | TPS: 1702 | 7797s
Step 1630/4000 | Loss: 2.3588 | LR: 0.000014 | TPS: 1702 | 7844s
Step 1640/4000 | Loss: 2.1927 | LR: 0.000014 | TPS: 1702 | 7892s
Step 1650/4000 | Loss: 1.9298 | LR: 0.000014 | TPS: 1703 | 7939s
Step 1660/4000 | Loss: 2.1604 | LR: 0.000014 | TPS: 1703 | 7986s
Step 1670/4000 | Loss: 2.0326 | LR: 0.000014 | TPS: 1703 | 8033s
Step 1680/4000 | Loss: 2.1872 | LR: 0.000014 | TPS: 1703 | 8080s
Step 1690/4000 | Loss: 2.0633 | LR: 0.000014 | TPS: 1703 | 8127s
Step 1700/4000 | Loss: 2.2547 | LR: 0.000014 | TPS: 1704 | 8174s
Step 1710/4000 | Loss: 1.8940 | LR: 0.000014 | TPS: 1704 | 8221s
Step 1720/4000 | Loss: 2.0726 | LR: 0.000014 | TPS: 1704 | 8269s
Step 1730/4000 | Loss: 2.0857 | LR: 0.000014 | TPS: 1704 | 8316s
Step 1740/4000 | Loss: 2.0686 | LR: 0.000014 | TPS: 1704 | 8363s
Step 1750/4000 | Loss: 2.1306 | LR: 0.000014 | TPS: 1705 | 8410s
Step 1760/4000 | Loss: 2.0932 | LR: 0.000013 | TPS: 1705 | 8457s
Step 1770/4000 | Loss: 2.0751 | LR: 0.000013 | TPS: 1705 | 8504s
Step 1780/4000 | Loss: 2.1802 | LR: 0.000013 | TPS: 1705 | 8551s
Step 1790/4000 | Loss: 1.6657 | LR: 0.000013 | TPS: 1705 | 8599s
Step 1800/4000 | Loss: 2.1290 | LR: 0.000013 | TPS: 1706 | 8646s
Step 1810/4000 | Loss: 2.1032 | LR: 0.000013 | TPS: 1706 | 8693s
Step 1820/4000 | Loss: 2.1255 | LR: 0.000013 | TPS: 1706 | 8740s
Step 1830/4000 | Loss: 2.1091 | LR: 0.000013 | TPS: 1706 | 8787s
Step 1840/4000 | Loss: 1.9875 | LR: 0.000013 | TPS: 1706 | 8834s
Step 1850/4000 | Loss: 1.9615 | LR: 0.000013 | TPS: 1706 | 8881s
Step 1860/4000 | Loss: 2.0189 | LR: 0.000013 | TPS: 1707 | 8929s
Step 1870/4000 | Loss: 2.1387 | LR: 0.000013 | TPS: 1707 | 8976s
Step 1880/4000 | Loss: 2.0963 | LR: 0.000013 | TPS: 1707 | 9023s
Step 1890/4000 | Loss: 2.1750 | LR: 0.000013 | TPS: 1707 | 9070s
Step 1900/4000 | Loss: 2.3945 | LR: 0.000012 | TPS: 1707 | 9117s
Step 1910/4000 | Loss: 2.1515 | LR: 0.000012 | TPS: 1707 | 9164s
Step 1920/4000 | Loss: 2.2224 | LR: 0.000012 | TPS: 1708 | 9211s
Step 1930/4000 | Loss: 2.3160 | LR: 0.000012 | TPS: 1708 | 9259s
Step 1940/4000 | Loss: 2.0126 | LR: 0.000012 | TPS: 1708 | 9306s
Step 1950/4000 | Loss: 2.2443 | LR: 0.000012 | TPS: 1708 | 9353s
Step 1960/4000 | Loss: 1.9590 | LR: 0.000012 | TPS: 1708 | 9400s
Step 1970/4000 | Loss: 2.2280 | LR: 0.000012 | TPS: 1708 | 9447s
Step 1980/4000 | Loss: 1.9723 | LR: 0.000012 | TPS: 1708 | 9494s
Step 1990/4000 | Loss: 2.0697 | LR: 0.000012 | TPS: 1709 | 9541s
Step 2000/4000 | Loss: 2.0568 | LR: 0.000012 | TPS: 1709 | 9589s
  📊 Val loss: 2.1674 

  🔤 Generation samples (step 2000):
    [EN] Paris (pronounced "Paris") is a city located in northeastern France. It borders Germany to the east, with Belgium and Luxembourg as its easternmost provinces.
    [HE] בצרפת, העיר העתיקה היא אזור התיירות העיקרי.
    [AR] باريس
    [FA] پاریس، پایتخت کشور فرانسه است.
    [TRANSLATE] The answer is YES.

Step 2010/4000 | Loss: 1.9474 | LR: 0.000012 | TPS: 1708 | 9643s
Step 2020/4000 | Loss: 2.1131 | LR: 0.000012 | TPS: 1708 | 9690s
Step 2030/4000 | Loss: 2.0446 | LR: 0.000012 | TPS: 1708 | 9737s
Step 2040/4000 | Loss: 2.2229 | LR: 0.000011 | TPS: 1708 | 9784s
Step 2050/4000 | Loss: 2.1576 | LR: 0.000011 | TPS: 1708 | 9832s
Step 2060/4000 | Loss: 2.1899 | LR: 0.000011 | TPS: 1708 | 9879s
Step 2070/4000 | Loss: 2.0957 | LR: 0.000011 | TPS: 1708 | 9926s
Step 2080/4000 | Loss: 2.2643 | LR: 0.000011 | TPS: 1709 | 9973s
Step 2090/4000 | Loss: 2.0676 | LR: 0.000011 | TPS: 1709 | 10020s
Step 2100/4000 | Loss: 2.1386 | LR: 0.000011 | TPS: 1709 | 10067s
Step 2110/4000 | Loss: 2.1891 | LR: 0.000011 | TPS: 1709 | 10114s
Step 2120/4000 | Loss: 1.9532 | LR: 0.000011 | TPS: 1709 | 10162s
Step 2130/4000 | Loss: 1.9766 | LR: 0.000011 | TPS: 1709 | 10209s
Step 2140/4000 | Loss: 2.3656 | LR: 0.000011 | TPS: 1709 | 10256s
Step 2150/4000 | Loss: 2.0545 | LR: 0.000011 | TPS: 1709 | 10303s
Step 2160/4000 | Loss: 1.9706 | LR: 0.000011 | TPS: 1710 | 10350s
Step 2170/4000 | Loss: 2.0302 | LR: 0.000010 | TPS: 1710 | 10397s
Step 2180/4000 | Loss: 2.1752 | LR: 0.000010 | TPS: 1710 | 10444s
Step 2190/4000 | Loss: 2.1455 | LR: 0.000010 | TPS: 1710 | 10492s
Step 2200/4000 | Loss: 2.2238 | LR: 0.000010 | TPS: 1710 | 10539s
Step 2210/4000 | Loss: 2.1010 | LR: 0.000010 | TPS: 1710 | 10586s
Step 2220/4000 | Loss: 2.1831 | LR: 0.000010 | TPS: 1710 | 10633s
Step 2230/4000 | Loss: 1.6542 | LR: 0.000010 | TPS: 1710 | 10680s
Step 2240/4000 | Loss: 2.1102 | LR: 0.000010 | TPS: 1711 | 10727s
Step 2250/4000 | Loss: 2.2099 | LR: 0.000010 | TPS: 1711 | 10774s
Step 2260/4000 | Loss: 2.1750 | LR: 0.000010 | TPS: 1711 | 10821s
Step 2270/4000 | Loss: 2.2369 | LR: 0.000010 | TPS: 1711 | 10869s
Step 2280/4000 | Loss: 2.0393 | LR: 0.000010 | TPS: 1711 | 10916s
Step 2290/4000 | Loss: 2.3140 | LR: 0.000010 | TPS: 1711 | 10963s
Step 2300/4000 | Loss: 2.0601 | LR: 0.000010 | TPS: 1711 | 11010s
Step 2310/4000 | Loss: 2.1472 | LR: 0.000009 | TPS: 1711 | 11057s
Step 2320/4000 | Loss: 2.0987 | LR: 0.000009 | TPS: 1712 | 11104s
Step 2330/4000 | Loss: 2.0354 | LR: 0.000009 | TPS: 1712 | 11152s
Step 2340/4000 | Loss: 1.9309 | LR: 0.000009 | TPS: 1712 | 11199s
Step 2350/4000 | Loss: 2.1222 | LR: 0.000009 | TPS: 1712 | 11246s
Step 2360/4000 | Loss: 1.9861 | LR: 0.000009 | TPS: 1712 | 11293s
Step 2370/4000 | Loss: 2.1986 | LR: 0.000009 | TPS: 1712 | 11340s
Step 2380/4000 | Loss: 2.0335 | LR: 0.000009 | TPS: 1712 | 11387s
Step 2390/4000 | Loss: 2.2123 | LR: 0.000009 | TPS: 1712 | 11434s
Step 2400/4000 | Loss: 2.0287 | LR: 0.000009 | TPS: 1712 | 11482s
  📊 Val loss: 2.1943 
Step 2410/4000 | Loss: 2.0483 | LR: 0.000009 | TPS: 1712 | 11534s
Step 2420/4000 | Loss: 2.0710 | LR: 0.000009 | TPS: 1712 | 11581s
Step 2430/4000 | Loss: 2.3005 | LR: 0.000009 | TPS: 1712 | 11629s
Step 2440/4000 | Loss: 2.0617 | LR: 0.000009 | TPS: 1712 | 11676s
Step 2450/4000 | Loss: 2.2063 | LR: 0.000008 | TPS: 1712 | 11723s
Step 2460/4000 | Loss: 2.0405 | LR: 0.000008 | TPS: 1712 | 11770s
Step 2470/4000 | Loss: 2.2280 | LR: 0.000008 | TPS: 1712 | 11817s
Step 2480/4000 | Loss: 2.3856 | LR: 0.000008 | TPS: 1712 | 11864s
Step 2490/4000 | Loss: 1.9853 | LR: 0.000008 | TPS: 1712 | 11911s
Step 2500/4000 | Loss: 2.0673 | LR: 0.000008 | TPS: 1713 | 11959s
Step 2510/4000 | Loss: 2.1777 | LR: 0.000008 | TPS: 1713 | 12006s
Step 2520/4000 | Loss: 1.9846 | LR: 0.000008 | TPS: 1713 | 12053s
Step 2530/4000 | Loss: 2.1922 | LR: 0.000008 | TPS: 1713 | 12100s
Step 2540/4000 | Loss: 2.0542 | LR: 0.000008 | TPS: 1713 | 12147s
Step 2550/4000 | Loss: 2.1041 | LR: 0.000008 | TPS: 1713 | 12194s
Step 2560/4000 | Loss: 2.0099 | LR: 0.000008 | TPS: 1713 | 12241s
Step 2570/4000 | Loss: 1.8186 | LR: 0.000008 | TPS: 1713 | 12289s
Step 2580/4000 | Loss: 2.2079 | LR: 0.000008 | TPS: 1713 | 12336s
Step 2590/4000 | Loss: 1.9931 | LR: 0.000007 | TPS: 1713 | 12383s
Step 2600/4000 | Loss: 2.0986 | LR: 0.000007 | TPS: 1714 | 12430s
Step 2610/4000 | Loss: 2.0439 | LR: 0.000007 | TPS: 1714 | 12477s
Step 2620/4000 | Loss: 1.9408 | LR: 0.000007 | TPS: 1714 | 12524s
Step 2630/4000 | Loss: 2.1992 | LR: 0.000007 | TPS: 1714 | 12571s
Step 2640/4000 | Loss: 2.0929 | LR: 0.000007 | TPS: 1714 | 12619s
Step 2650/4000 | Loss: 1.9728 | LR: 0.000007 | TPS: 1714 | 12666s
Step 2660/4000 | Loss: 1.8369 | LR: 0.000007 | TPS: 1714 | 12713s
Step 2670/4000 | Loss: 1.9926 | LR: 0.000007 | TPS: 1714 | 12760s
Step 2680/4000 | Loss: 2.0414 | LR: 0.000007 | TPS: 1714 | 12807s
Step 2690/4000 | Loss: 2.1368 | LR: 0.000007 | TPS: 1714 | 12854s
Step 2700/4000 | Loss: 2.0254 | LR: 0.000007 | TPS: 1714 | 12901s
Step 2710/4000 | Loss: 2.1572 | LR: 0.000007 | TPS: 1715 | 12948s
Step 2720/4000 | Loss: 2.0418 | LR: 0.000007 | TPS: 1715 | 12996s
Step 2730/4000 | Loss: 2.1235 | LR: 0.000007 | TPS: 1715 | 13043s
Step 2740/4000 | Loss: 2.0756 | LR: 0.000006 | TPS: 1715 | 13090s
Step 2750/4000 | Loss: 2.1417 | LR: 0.000006 | TPS: 1715 | 13137s
Step 2760/4000 | Loss: 1.9427 | LR: 0.000006 | TPS: 1715 | 13184s
Step 2770/4000 | Loss: 2.1166 | LR: 0.000006 | TPS: 1715 | 13231s
Step 2780/4000 | Loss: 1.9711 | LR: 0.000006 | TPS: 1715 | 13278s
Step 2790/4000 | Loss: 2.1390 | LR: 0.000006 | TPS: 1715 | 13326s
Step 2800/4000 | Loss: 2.0557 | LR: 0.000006 | TPS: 1715 | 13373s
  📊 Val loss: 2.1839 
Step 2810/4000 | Loss: 2.0581 | LR: 0.000006 | TPS: 1715 | 13425s
Step 2820/4000 | Loss: 2.1139 | LR: 0.000006 | TPS: 1715 | 13473s
Step 2830/4000 | Loss: 2.1228 | LR: 0.000006 | TPS: 1715 | 13520s
Step 2840/4000 | Loss: 1.9685 | LR: 0.000006 | TPS: 1715 | 13567s
Step 2850/4000 | Loss: 2.1206 | LR: 0.000006 | TPS: 1715 | 13614s
Step 2860/4000 | Loss: 2.1942 | LR: 0.000006 | TPS: 1715 | 13661s
Step 2870/4000 | Loss: 1.9068 | LR: 0.000006 | TPS: 1715 | 13708s
Step 2880/4000 | Loss: 2.2099 | LR: 0.000006 | TPS: 1715 | 13755s
Step 2890/4000 | Loss: 2.0948 | LR: 0.000006 | TPS: 1715 | 13803s
Step 2900/4000 | Loss: 2.0630 | LR: 0.000005 | TPS: 1715 | 13850s
Step 2910/4000 | Loss: 1.9867 | LR: 0.000005 | TPS: 1715 | 13897s
Step 2920/4000 | Loss: 2.0602 | LR: 0.000005 | TPS: 1715 | 13944s
Step 2930/4000 | Loss: 2.0163 | LR: 0.000005 | TPS: 1716 | 13991s
Step 2940/4000 | Loss: 2.0337 | LR: 0.000005 | TPS: 1716 | 14038s
Step 2950/4000 | Loss: 2.2476 | LR: 0.000005 | TPS: 1716 | 14085s
Step 2960/4000 | Loss: 2.0430 | LR: 0.000005 | TPS: 1716 | 14133s
Step 2970/4000 | Loss: 2.3037 | LR: 0.000005 | TPS: 1716 | 14180s
Step 2980/4000 | Loss: 2.0831 | LR: 0.000005 | TPS: 1716 | 14227s
Step 2990/4000 | Loss: 2.1781 | LR: 0.000005 | TPS: 1716 | 14274s
Step 3000/4000 | Loss: 2.0784 | LR: 0.000005 | TPS: 1716 | 14321s

  🔤 Generation samples (step 3000):
    [EN] The city of Paris is a metropolitan area in Europe, consisting of 57 counties. Its main cities include Lyons, Bordeaux and Valence.
    [HE] איטליה.
    [AR] باريس.
    [FA] پاریس پایتخت کشور فرانسه و یکی از شهرهای بزرگ این کشور است. شهر پاریس در شمال غربی قاره اروپا قرار دارد.
    [TRANSLATE] You are the first one in the world to learn how to think.

Step 3010/4000 | Loss: 2.1244 | LR: 0.000005 | TPS: 1716 | 14370s
Step 3020/4000 | Loss: 2.1107 | LR: 0.000005 | TPS: 1716 | 14417s
Step 3030/4000 | Loss: 2.3589 | LR: 0.000005 | TPS: 1716 | 14464s
Step 3040/4000 | Loss: 2.0592 | LR: 0.000005 | TPS: 1716 | 14511s
Step 3050/4000 | Loss: 2.0730 | LR: 0.000005 | TPS: 1716 | 14559s
Step 3060/4000 | Loss: 2.1365 | LR: 0.000005 | TPS: 1716 | 14606s
Step 3070/4000 | Loss: 1.9819 | LR: 0.000005 | TPS: 1716 | 14653s
Step 3080/4000 | Loss: 2.2175 | LR: 0.000004 | TPS: 1716 | 14700s
Step 3090/4000 | Loss: 2.1442 | LR: 0.000004 | TPS: 1716 | 14747s
Step 3100/4000 | Loss: 2.0811 | LR: 0.000004 | TPS: 1717 | 14794s
Step 3110/4000 | Loss: 2.1427 | LR: 0.000004 | TPS: 1717 | 14841s
Step 3120/4000 | Loss: 2.1722 | LR: 0.000004 | TPS: 1717 | 14889s
Step 3130/4000 | Loss: 2.0577 | LR: 0.000004 | TPS: 1717 | 14936s
Step 3140/4000 | Loss: 2.0873 | LR: 0.000004 | TPS: 1717 | 14983s
Step 3150/4000 | Loss: 2.2920 | LR: 0.000004 | TPS: 1717 | 15030s
Step 3160/4000 | Loss: 1.8839 | LR: 0.000004 | TPS: 1717 | 15077s
Step 3170/4000 | Loss: 2.0144 | LR: 0.000004 | TPS: 1717 | 15124s
Step 3180/4000 | Loss: 1.9689 | LR: 0.000004 | TPS: 1717 | 15171s
Step 3190/4000 | Loss: 2.2123 | LR: 0.000004 | TPS: 1717 | 15219s
Step 3200/4000 | Loss: 2.0510 | LR: 0.000004 | TPS: 1717 | 15266s
  📊 Val loss: 2.1269 
Step 3210/4000 | Loss: 2.4087 | LR: 0.000004 | TPS: 1717 | 15318s
Step 3220/4000 | Loss: 2.2608 | LR: 0.000004 | TPS: 1717 | 15365s
Step 3230/4000 | Loss: 2.1930 | LR: 0.000004 | TPS: 1717 | 15413s
Step 3240/4000 | Loss: 2.0713 | LR: 0.000004 | TPS: 1717 | 15460s
Step 3250/4000 | Loss: 2.2660 | LR: 0.000004 | TPS: 1717 | 15507s
Step 3260/4000 | Loss: 1.9479 | LR: 0.000004 | TPS: 1717 | 15554s
Step 3270/4000 | Loss: 1.9657 | LR: 0.000004 | TPS: 1717 | 15601s
Step 3280/4000 | Loss: 2.1884 | LR: 0.000004 | TPS: 1717 | 15648s
Step 3290/4000 | Loss: 2.0927 | LR: 0.000004 | TPS: 1717 | 15695s
Step 3300/4000 | Loss: 2.0393 | LR: 0.000003 | TPS: 1717 | 15743s
Step 3310/4000 | Loss: 2.1302 | LR: 0.000003 | TPS: 1717 | 15790s
Step 3320/4000 | Loss: 2.0059 | LR: 0.000003 | TPS: 1717 | 15837s
Step 3330/4000 | Loss: 1.8687 | LR: 0.000003 | TPS: 1717 | 15884s
Step 3340/4000 | Loss: 2.0293 | LR: 0.000003 | TPS: 1717 | 15931s
Step 3350/4000 | Loss: 2.1500 | LR: 0.000003 | TPS: 1718 | 15978s
Step 3360/4000 | Loss: 1.9667 | LR: 0.000003 | TPS: 1718 | 16025s
Step 3370/4000 | Loss: 2.1206 | LR: 0.000003 | TPS: 1718 | 16073s
Step 3380/4000 | Loss: 2.3028 | LR: 0.000003 | TPS: 1718 | 16120s
Step 3390/4000 | Loss: 2.0075 | LR: 0.000003 | TPS: 1718 | 16167s
Step 3400/4000 | Loss: 2.0562 | LR: 0.000003 | TPS: 1718 | 16214s
Step 3410/4000 | Loss: 1.9977 | LR: 0.000003 | TPS: 1718 | 16261s
Step 3420/4000 | Loss: 2.1680 | LR: 0.000003 | TPS: 1718 | 16308s
Step 3430/4000 | Loss: 2.0009 | LR: 0.000003 | TPS: 1718 | 16355s
Step 3440/4000 | Loss: 1.8301 | LR: 0.000003 | TPS: 1718 | 16403s
Step 3450/4000 | Loss: 2.0239 | LR: 0.000003 | TPS: 1718 | 16450s
Step 3460/4000 | Loss: 2.0535 | LR: 0.000003 | TPS: 1718 | 16497s
Step 3470/4000 | Loss: 2.1348 | LR: 0.000003 | TPS: 1718 | 16544s
Step 3480/4000 | Loss: 2.0337 | LR: 0.000003 | TPS: 1718 | 16591s
Step 3490/4000 | Loss: 1.9342 | LR: 0.000003 | TPS: 1718 | 16638s
Step 3500/4000 | Loss: 2.0052 | LR: 0.000003 | TPS: 1718 | 16685s
Step 3510/4000 | Loss: 1.9902 | LR: 0.000003 | TPS: 1718 | 16732s
Step 3520/4000 | Loss: 2.1567 | LR: 0.000003 | TPS: 1719 | 16780s
Step 3530/4000 | Loss: 2.0515 | LR: 0.000003 | TPS: 1719 | 16827s
Step 3540/4000 | Loss: 2.1572 | LR: 0.000003 | TPS: 1719 | 16874s
Step 3550/4000 | Loss: 2.1381 | LR: 0.000003 | TPS: 1719 | 16921s
Step 3560/4000 | Loss: 2.0383 | LR: 0.000003 | TPS: 1719 | 16968s
Step 3570/4000 | Loss: 2.3566 | LR: 0.000003 | TPS: 1719 | 17015s
Step 3580/4000 | Loss: 1.9773 | LR: 0.000003 | TPS: 1719 | 17062s
Step 3590/4000 | Loss: 2.0418 | LR: 0.000003 | TPS: 1719 | 17110s
Step 3600/4000 | Loss: 2.1756 | LR: 0.000002 | TPS: 1719 | 17157s
  📊 Val loss: 2.1478 
Step 3610/4000 | Loss: 2.0761 | LR: 0.000002 | TPS: 1718 | 17209s
Step 3620/4000 | Loss: 2.1353 | LR: 0.000002 | TPS: 1718 | 17257s
Step 3630/4000 | Loss: 2.1856 | LR: 0.000002 | TPS: 1719 | 17304s
Step 3640/4000 | Loss: 2.1298 | LR: 0.000002 | TPS: 1719 | 17351s
Step 3650/4000 | Loss: 2.0784 | LR: 0.000002 | TPS: 1719 | 17398s
Step 3660/4000 | Loss: 2.0533 | LR: 0.000002 | TPS: 1719 | 17445s
Step 3670/4000 | Loss: 2.2151 | LR: 0.000002 | TPS: 1719 | 17492s
Step 3680/4000 | Loss: 2.0177 | LR: 0.000002 | TPS: 1719 | 17539s
Step 3690/4000 | Loss: 2.1048 | LR: 0.000002 | TPS: 1719 | 17587s
Step 3700/4000 | Loss: 2.0629 | LR: 0.000002 | TPS: 1719 | 17634s
Step 3710/4000 | Loss: 2.0375 | LR: 0.000002 | TPS: 1719 | 17681s
Step 3720/4000 | Loss: 2.2282 | LR: 0.000002 | TPS: 1719 | 17728s
Step 3730/4000 | Loss: 2.2049 | LR: 0.000002 | TPS: 1719 | 17775s
Step 3740/4000 | Loss: 2.0247 | LR: 0.000002 | TPS: 1719 | 17822s
Step 3750/4000 | Loss: 2.0337 | LR: 0.000002 | TPS: 1719 | 17869s
Step 3760/4000 | Loss: 2.0922 | LR: 0.000002 | TPS: 1719 | 17917s
Step 3770/4000 | Loss: 2.1018 | LR: 0.000002 | TPS: 1719 | 17964s
Step 3780/4000 | Loss: 2.1183 | LR: 0.000002 | TPS: 1719 | 18011s
Step 3790/4000 | Loss: 2.2469 | LR: 0.000002 | TPS: 1719 | 18058s
Step 3800/4000 | Loss: 2.1373 | LR: 0.000002 | TPS: 1719 | 18105s
Step 3810/4000 | Loss: 2.1103 | LR: 0.000002 | TPS: 1719 | 18152s
Step 3820/4000 | Loss: 2.0317 | LR: 0.000002 | TPS: 1719 | 18199s
Step 3830/4000 | Loss: 2.0022 | LR: 0.000002 | TPS: 1720 | 18247s
Step 3840/4000 | Loss: 2.1618 | LR: 0.000002 | TPS: 1720 | 18294s
Step 3850/4000 | Loss: 2.1421 | LR: 0.000002 | TPS: 1720 | 18341s
Step 3860/4000 | Loss: 1.9279 | LR: 0.000002 | TPS: 1720 | 18388s
Step 3870/4000 | Loss: 2.1657 | LR: 0.000002 | TPS: 1720 | 18435s
Step 3880/4000 | Loss: 2.1433 | LR: 0.000002 | TPS: 1720 | 18482s
Step 3890/4000 | Loss: 2.0893 | LR: 0.000002 | TPS: 1720 | 18529s
Step 3900/4000 | Loss: 2.0036 | LR: 0.000002 | TPS: 1720 | 18576s
Step 3910/4000 | Loss: 2.0691 | LR: 0.000002 | TPS: 1720 | 18624s
Step 3920/4000 | Loss: 2.0282 | LR: 0.000002 | TPS: 1720 | 18671s
Step 3930/4000 | Loss: 1.9818 | LR: 0.000002 | TPS: 1720 | 18718s
Step 3940/4000 | Loss: 2.1466 | LR: 0.000002 | TPS: 1720 | 18765s
Step 3950/4000 | Loss: 2.0455 | LR: 0.000002 | TPS: 1720 | 18812s
Step 3960/4000 | Loss: 2.1226 | LR: 0.000002 | TPS: 1720 | 18859s
Step 3970/4000 | Loss: 1.9890 | LR: 0.000002 | TPS: 1720 | 18906s
Step 3980/4000 | Loss: 2.1891 | LR: 0.000002 | TPS: 1720 | 18954s
Step 3990/4000 | Loss: 1.8920 | LR: 0.000002 | TPS: 1720 | 19001s
Step 4000/4000 | Loss: 2.0073 | LR: 0.000002 | TPS: 1720 | 19048s
  📊 Val loss: 2.1472 

  🔤 Generation samples (step 4000):
    [EN] The capital of France consists of 38 cities, 26.9% (14) of which are in the metropolitan area.
    [HE] צרפת היא אחת מיעדי התיירות הפופולאריים ביותר בעולם, בשל היותה מוקד משיכה תיירותי משמעותי עבור תיירים מכל רחבי העולם. העיר בנויה משני חלקים עיקריים - כיכר ד'ארסאן (Droite Sud) ורחוב ד'ארסאן (De La Roch
    [AR] باريس.
    [FA] پاریس شهری بزرگ و تاریخی در شمال غربی اروپا است.
    [TRANSLATE] It’s very short.


============================================================
SFT TRAINING COMPLETE
Steps: 4000, Time: 19057s (317.6min)
Best val loss: 2.1164
Model saved to: /tmp/sft/sft_model_v2.pt
============================================================
Uploading to S3...