LEMA-llama-2-7b / training_logs.txt
Pomilon's picture
Create training_logs.txt
9a58d27 verified
619.1s 534 2026-02-16 12:48:08,993 - __main__ - INFO - Step 10/625 | Loss: 2.1732 | VRAM: 6.36GB | RAM: 2.39GB
619.1s 535 INFO:__main__:Step 10/625 | Loss: 2.1732 | VRAM: 6.36GB | RAM: 2.39GB
941.4s 536 2026-02-16 12:53:31,274 - __main__ - INFO - Step 20/625 | Loss: 1.1008 | VRAM: 6.36GB | RAM: 2.40GB
941.4s 537 INFO:__main__:Step 20/625 | Loss: 1.1008 | VRAM: 6.36GB | RAM: 2.40GB
1263.7s 538 2026-02-16 12:58:53,631 - __main__ - INFO - Step 30/625 | Loss: 0.6457 | VRAM: 6.36GB | RAM: 2.42GB
1263.7s 539 INFO:__main__:Step 30/625 | Loss: 0.6457 | VRAM: 6.36GB | RAM: 2.42GB
1586.1s 540 2026-02-16 13:04:16,041 - __main__ - INFO - Step 40/625 | Loss: 0.4113 | VRAM: 6.36GB | RAM: 2.39GB
1586.1s 541 INFO:__main__:Step 40/625 | Loss: 0.4113 | VRAM: 6.36GB | RAM: 2.39GB
1908.6s 542 2026-02-16 13:09:38,514 - __main__ - INFO - Step 50/625 | Loss: 0.5867 | VRAM: 6.36GB | RAM: 2.40GB
1908.6s 543 INFO:__main__:Step 50/625 | Loss: 0.5867 | VRAM: 6.36GB | RAM: 2.40GB
2231.0s 544 2026-02-16 13:15:00,931 - __main__ - INFO - Step 60/625 | Loss: 0.2535 | VRAM: 6.36GB | RAM: 2.39GB
2231.0s 545 INFO:__main__:Step 60/625 | Loss: 0.2535 | VRAM: 6.36GB | RAM: 2.39GB
2553.5s 546 2026-02-16 13:20:23,403 - __main__ - INFO - Step 70/625 | Loss: 0.4313 | VRAM: 6.36GB | RAM: 2.37GB
2553.5s 547 INFO:__main__:Step 70/625 | Loss: 0.4313 | VRAM: 6.36GB | RAM: 2.37GB
2875.9s 548 2026-02-16 13:25:45,819 - __main__ - INFO - Step 80/625 | Loss: 0.1017 | VRAM: 6.36GB | RAM: 2.36GB
2875.9s 549 INFO:__main__:Step 80/625 | Loss: 0.1017 | VRAM: 6.36GB | RAM: 2.36GB
3198.3s 550 2026-02-16 13:31:08,168 - __main__ - INFO - Step 90/625 | Loss: 0.1827 | VRAM: 6.36GB | RAM: 2.37GB
3198.3s 551 INFO:__main__:Step 90/625 | Loss: 0.1827 | VRAM: 6.36GB | RAM: 2.37GB
3520.6s 552 2026-02-16 13:36:30,500 - __main__ - INFO - Step 100/625 | Loss: 0.0677 | VRAM: 6.36GB | RAM: 2.39GB
3520.6s 553 INFO:__main__:Step 100/625 | Loss: 0.0677 | VRAM: 6.36GB | RAM: 2.39GB
3843.0s 554 2026-02-16 13:41:52,842 - __main__ - INFO - Step 110/625 | Loss: 0.0511 | VRAM: 6.36GB | RAM: 2.36GB
3843.0s 555 INFO:__main__:Step 110/625 | Loss: 0.0511 | VRAM: 6.36GB | RAM: 2.36GB
4165.3s 556 2026-02-16 13:47:15,184 - __main__ - INFO - Step 120/625 | Loss: 0.0565 | VRAM: 6.36GB | RAM: 2.37GB
4165.3s 557 INFO:__main__:Step 120/625 | Loss: 0.0565 | VRAM: 6.36GB | RAM: 2.37GB
4487.6s 558 2026-02-16 13:52:37,493 - __main__ - INFO - Step 130/625 | Loss: 0.0511 | VRAM: 6.36GB | RAM: 2.36GB
4487.6s 559 INFO:__main__:Step 130/625 | Loss: 0.0511 | VRAM: 6.36GB | RAM: 2.36GB
4810.0s 560 2026-02-16 13:57:59,925 - __main__ - INFO - Step 140/625 | Loss: 0.0769 | VRAM: 6.36GB | RAM: 2.36GB
4810.0s 561 INFO:__main__:Step 140/625 | Loss: 0.0769 | VRAM: 6.36GB | RAM: 2.36GB
5132.4s 562 2026-02-16 14:03:22,277 - __main__ - INFO - Step 150/625 | Loss: 0.0484 | VRAM: 6.36GB | RAM: 2.35GB
5132.4s 563 INFO:__main__:Step 150/625 | Loss: 0.0484 | VRAM: 6.36GB | RAM: 2.35GB
5454.8s 564 2026-02-16 14:08:44,697 - __main__ - INFO - Step 160/625 | Loss: 0.0401 | VRAM: 6.36GB | RAM: 2.37GB
5454.8s 565 INFO:__main__:Step 160/625 | Loss: 0.0401 | VRAM: 6.36GB | RAM: 2.37GB
5777.2s 566 2026-02-16 14:14:07,061 - __main__ - INFO - Step 170/625 | Loss: 0.0533 | VRAM: 6.36GB | RAM: 2.35GB
5777.2s 567 INFO:__main__:Step 170/625 | Loss: 0.0533 | VRAM: 6.36GB | RAM: 2.35GB
6099.6s 568 2026-02-16 14:19:29,445 - __main__ - INFO - Step 180/625 | Loss: 0.0391 | VRAM: 6.36GB | RAM: 2.37GB
6099.6s 569 INFO:__main__:Step 180/625 | Loss: 0.0391 | VRAM: 6.36GB | RAM: 2.37GB
6421.9s 570 2026-02-16 14:24:51,789 - __main__ - INFO - Step 190/625 | Loss: 0.0464 | VRAM: 6.36GB | RAM: 2.37GB
6421.9s 571 INFO:__main__:Step 190/625 | Loss: 0.0464 | VRAM: 6.36GB | RAM: 2.37GB
6744.3s 572 2026-02-16 14:30:14,166 - __main__ - INFO - Step 200/625 | Loss: 0.0462 | VRAM: 6.36GB | RAM: 2.37GB
6744.3s 573 INFO:__main__:Step 200/625 | Loss: 0.0462 | VRAM: 6.36GB | RAM: 2.37GB
7066.7s 574 2026-02-16 14:35:36,596 - __main__ - INFO - Step 210/625 | Loss: 0.0450 | VRAM: 6.36GB | RAM: 2.38GB
7066.7s 575 INFO:__main__:Step 210/625 | Loss: 0.0450 | VRAM: 6.36GB | RAM: 2.38GB
7389.1s 576 2026-02-16 14:40:58,993 - __main__ - INFO - Step 220/625 | Loss: 0.0401 | VRAM: 6.36GB | RAM: 2.37GB
7389.1s 577 INFO:__main__:Step 220/625 | Loss: 0.0401 | VRAM: 6.36GB | RAM: 2.37GB
7711.5s 578 2026-02-16 14:46:21,447 - __main__ - INFO - Step 230/625 | Loss: 0.0433 | VRAM: 6.36GB | RAM: 2.38GB
7711.5s 579 INFO:__main__:Step 230/625 | Loss: 0.0433 | VRAM: 6.36GB | RAM: 2.38GB
8033.9s 580 2026-02-16 14:51:43,849 - __main__ - INFO - Step 240/625 | Loss: 0.0525 | VRAM: 6.36GB | RAM: 2.38GB
8033.9s 581 INFO:__main__:Step 240/625 | Loss: 0.0525 | VRAM: 6.36GB | RAM: 2.38GB
8356.3s 582 2026-02-16 14:57:06,160 - __main__ - INFO - Step 250/625 | Loss: 0.0464 | VRAM: 6.36GB | RAM: 2.39GB
8356.3s 583 INFO:__main__:Step 250/625 | Loss: 0.0464 | VRAM: 6.36GB | RAM: 2.39GB
8678.6s 584 2026-02-16 15:02:28,505 - __main__ - INFO - Step 260/625 | Loss: 0.0515 | VRAM: 6.36GB | RAM: 2.38GB
8678.6s 585 INFO:__main__:Step 260/625 | Loss: 0.0515 | VRAM: 6.36GB | RAM: 2.38GB
9000.9s 586 2026-02-16 15:07:50,785 - __main__ - INFO - Step 270/625 | Loss: 0.0405 | VRAM: 6.36GB | RAM: 2.39GB
9000.9s 587 INFO:__main__:Step 270/625 | Loss: 0.0405 | VRAM: 6.36GB | RAM: 2.39GB
9323.2s 588 2026-02-16 15:13:13,084 - __main__ - INFO - Step 280/625 | Loss: 0.0395 | VRAM: 6.36GB | RAM: 2.39GB
9323.2s 589 INFO:__main__:Step 280/625 | Loss: 0.0395 | VRAM: 6.36GB | RAM: 2.39GB
9645.5s 590 2026-02-16 15:18:35,374 - __main__ - INFO - Step 290/625 | Loss: 0.0413 | VRAM: 6.36GB | RAM: 2.38GB
9645.5s 591 INFO:__main__:Step 290/625 | Loss: 0.0413 | VRAM: 6.36GB | RAM: 2.38GB
9967.8s 592 2026-02-16 15:23:57,658 - __main__ - INFO - Step 300/625 | Loss: 0.0407 | VRAM: 6.36GB | RAM: 2.38GB
9967.8s 593 INFO:__main__:Step 300/625 | Loss: 0.0407 | VRAM: 6.36GB | RAM: 2.38GB
10290.0s 594 2026-02-16 15:29:19,910 - __main__ - INFO - Step 310/625 | Loss: 0.0462 | VRAM: 6.36GB | RAM: 2.39GB
10290.0s 595 INFO:__main__:Step 310/625 | Loss: 0.0462 | VRAM: 6.36GB | RAM: 2.39GB
10612.4s 596 2026-02-16 15:34:42,316 - __main__ - INFO - Step 320/625 | Loss: 0.0482 | VRAM: 6.36GB | RAM: 2.39GB
10612.4s 597 INFO:__main__:Step 320/625 | Loss: 0.0482 | VRAM: 6.36GB | RAM: 2.39GB
10934.8s 598 2026-02-16 15:40:04,673 - __main__ - INFO - Step 330/625 | Loss: 0.0432 | VRAM: 6.36GB | RAM: 2.39GB
10934.8s 599 INFO:__main__:Step 330/625 | Loss: 0.0432 | VRAM: 6.36GB | RAM: 2.39GB
11257.1s 600 2026-02-16 15:45:26,987 - __main__ - INFO - Step 340/625 | Loss: 0.0479 | VRAM: 6.36GB | RAM: 2.38GB
11257.1s 601 INFO:__main__:Step 340/625 | Loss: 0.0479 | VRAM: 6.36GB | RAM: 2.38GB
11579.4s 602 2026-02-16 15:50:49,273 - __main__ - INFO - Step 350/625 | Loss: 0.0427 | VRAM: 6.36GB | RAM: 2.39GB
11579.4s 603 INFO:__main__:Step 350/625 | Loss: 0.0427 | VRAM: 6.36GB | RAM: 2.39GB
11901.6s 604 2026-02-16 15:56:11,550 - __main__ - INFO - Step 360/625 | Loss: 0.0373 | VRAM: 6.36GB | RAM: 2.39GB
11901.6s 605 INFO:__main__:Step 360/625 | Loss: 0.0373 | VRAM: 6.36GB | RAM: 2.39GB
12224.0s 606 2026-02-16 16:01:33,889 - __main__ - INFO - Step 370/625 | Loss: 0.0440 | VRAM: 6.36GB | RAM: 2.38GB
12224.0s 607 INFO:__main__:Step 370/625 | Loss: 0.0440 | VRAM: 6.36GB | RAM: 2.38GB
12546.4s 608 2026-02-16 16:06:56,280 - __main__ - INFO - Step 380/625 | Loss: 0.0445 | VRAM: 6.36GB | RAM: 2.36GB
12546.4s 609 INFO:__main__:Step 380/625 | Loss: 0.0445 | VRAM: 6.36GB | RAM: 2.36GB
12868.7s 610 2026-02-16 16:12:18,600 - __main__ - INFO - Step 390/625 | Loss: 0.0427 | VRAM: 6.36GB | RAM: 2.39GB
12868.7s 611 INFO:__main__:Step 390/625 | Loss: 0.0427 | VRAM: 6.36GB | RAM: 2.39GB
13191.0s 612 2026-02-16 16:17:40,899 - __main__ - INFO - Step 400/625 | Loss: 0.0412 | VRAM: 6.36GB | RAM: 2.39GB
13191.0s 613 INFO:__main__:Step 400/625 | Loss: 0.0412 | VRAM: 6.36GB | RAM: 2.39GB
13513.3s 614 2026-02-16 16:23:03,243 - __main__ - INFO - Step 410/625 | Loss: 0.0431 | VRAM: 6.36GB | RAM: 2.39GB
13513.3s 615 INFO:__main__:Step 410/625 | Loss: 0.0431 | VRAM: 6.36GB | RAM: 2.39GB
13835.6s 616 2026-02-16 16:28:25,499 - __main__ - INFO - Step 420/625 | Loss: 0.0427 | VRAM: 6.36GB | RAM: 2.39GB
13835.6s 617 INFO:__main__:Step 420/625 | Loss: 0.0427 | VRAM: 6.36GB | RAM: 2.39GB
14158.0s 618 2026-02-16 16:33:47,864 - __main__ - INFO - Step 430/625 | Loss: 0.0464 | VRAM: 6.36GB | RAM: 2.39GB
14158.0s 619 INFO:__main__:Step 430/625 | Loss: 0.0464 | VRAM: 6.36GB | RAM: 2.39GB
14480.4s 620 2026-02-16 16:39:10,223 - __main__ - INFO - Step 440/625 | Loss: 0.0423 | VRAM: 6.36GB | RAM: 2.36GB
14480.4s 621 INFO:__main__:Step 440/625 | Loss: 0.0423 | VRAM: 6.36GB | RAM: 2.36GB
14802.6s 622 2026-02-16 16:44:32,512 - __main__ - INFO - Step 450/625 | Loss: 0.0435 | VRAM: 6.36GB | RAM: 2.37GB
14802.6s 623 INFO:__main__:Step 450/625 | Loss: 0.0435 | VRAM: 6.36GB | RAM: 2.37GB
15124.9s 624 2026-02-16 16:49:54,804 - __main__ - INFO - Step 460/625 | Loss: 0.0372 | VRAM: 6.36GB | RAM: 2.38GB
15124.9s 625 INFO:__main__:Step 460/625 | Loss: 0.0372 | VRAM: 6.36GB | RAM: 2.38GB
15447.2s 626 2026-02-16 16:55:17,098 - __main__ - INFO - Step 470/625 | Loss: 0.0371 | VRAM: 6.36GB | RAM: 2.38GB
15447.2s 627 INFO:__main__:Step 470/625 | Loss: 0.0371 | VRAM: 6.36GB | RAM: 2.38GB
15769.5s 628 2026-02-16 17:00:39,362 - __main__ - INFO - Step 480/625 | Loss: 0.0410 | VRAM: 6.36GB | RAM: 2.39GB
15769.5s 629 INFO:__main__:Step 480/625 | Loss: 0.0410 | VRAM: 6.36GB | RAM: 2.39GB
16091.9s 630 2026-02-16 17:06:01,743 - __main__ - INFO - Step 490/625 | Loss: 0.0430 | VRAM: 6.36GB | RAM: 2.36GB
16091.9s 631 INFO:__main__:Step 490/625 | Loss: 0.0430 | VRAM: 6.36GB | RAM: 2.36GB
16415.6s 632 2026-02-16 17:11:25,521 - __main__ - INFO - Step 500/625 | Loss: 0.0459 | VRAM: 6.36GB | RAM: 2.52GB
16415.6s 633 INFO:__main__:Step 500/625 | Loss: 0.0459 | VRAM: 6.36GB | RAM: 2.52GB
16737.4s 634 2026-02-16 17:16:47,226 - __main__ - INFO - Step 510/625 | Loss: 0.0408 | VRAM: 6.36GB | RAM: 2.39GB
16737.4s 635 INFO:__main__:Step 510/625 | Loss: 0.0408 | VRAM: 6.36GB | RAM: 2.39GB
17059.6s 636 2026-02-16 17:22:09,481 - __main__ - INFO - Step 520/625 | Loss: 0.0410 | VRAM: 6.36GB | RAM: 2.39GB
17059.6s 637 INFO:__main__:Step 520/625 | Loss: 0.0410 | VRAM: 6.36GB | RAM: 2.39GB
17381.8s 638 2026-02-16 17:27:31,724 - __main__ - INFO - Step 530/625 | Loss: 0.0432 | VRAM: 6.36GB | RAM: 2.39GB
17381.8s 639 INFO:__main__:Step 530/625 | Loss: 0.0432 | VRAM: 6.36GB | RAM: 2.39GB
17704.2s 640 2026-02-16 17:32:54,035 - __main__ - INFO - Step 540/625 | Loss: 0.0413 | VRAM: 6.36GB | RAM: 2.38GB
17704.2s 641 INFO:__main__:Step 540/625 | Loss: 0.0413 | VRAM: 6.36GB | RAM: 2.38GB
18026.4s 642 2026-02-16 17:38:16,344 - __main__ - INFO - Step 550/625 | Loss: 0.0386 | VRAM: 6.36GB | RAM: 2.39GB
18026.4s 643 INFO:__main__:Step 550/625 | Loss: 0.0386 | VRAM: 6.36GB | RAM: 2.39GB
18348.7s 644 2026-02-16 17:43:38,609 - __main__ - INFO - Step 560/625 | Loss: 0.0391 | VRAM: 6.36GB | RAM: 2.38GB
18348.7s 645 INFO:__main__:Step 560/625 | Loss: 0.0391 | VRAM: 6.36GB | RAM: 2.38GB
18671.0s 646 2026-02-16 17:49:00,843 - __main__ - INFO - Step 570/625 | Loss: 0.0387 | VRAM: 6.36GB | RAM: 2.38GB
18671.0s 647 INFO:__main__:Step 570/625 | Loss: 0.0387 | VRAM: 6.36GB | RAM: 2.38GB
18993.2s 648 2026-02-16 17:54:23,093 - __main__ - INFO - Step 580/625 | Loss: 0.0470 | VRAM: 6.36GB | RAM: 2.38GB
18993.2s 649 INFO:__main__:Step 580/625 | Loss: 0.0470 | VRAM: 6.36GB | RAM: 2.38GB
19315.4s 650 2026-02-16 17:59:45,309 - __main__ - INFO - Step 590/625 | Loss: 0.0377 | VRAM: 6.36GB | RAM: 2.38GB
19315.4s 651 INFO:__main__:Step 590/625 | Loss: 0.0377 | VRAM: 6.36GB | RAM: 2.38GB
19637.6s 652 2026-02-16 18:05:07,525 - __main__ - INFO - Step 600/625 | Loss: 0.0406 | VRAM: 6.36GB | RAM: 2.38GB
19637.6s 653 INFO:__main__:Step 600/625 | Loss: 0.0406 | VRAM: 6.36GB | RAM: 2.38GB
19959.8s 654 2026-02-16 18:10:29,713 - __main__ - INFO - Step 610/625 | Loss: 0.0399 | VRAM: 6.36GB | RAM: 2.39GB
19959.8s 655 INFO:__main__:Step 610/625 | Loss: 0.0399 | VRAM: 6.36GB | RAM: 2.39GB
20282.0s 656 2026-02-16 18:15:51,872 - __main__ - INFO - Step 620/625 | Loss: 0.0388 | VRAM: 6.36GB | RAM: 2.38GB
20282.0s 657 INFO:__main__:Step 620/625 | Loss: 0.0388 | VRAM: 6.36GB | RAM: 2.38GB
20443.1s 658 2026-02-16 18:18:32,947 - __main__ - INFO - Training complete.
20443.1s 659 INFO:__main__:Training complete.
20443.1s 660 2026-02-16 18:18:32,947 - __main__ - INFO - Saving final model to checkpoints/final
20443.1s 661 INFO:__main__:Saving final model to checkpoints/final
20444.6s 662 2026-02-16 18:18:34,524 - __main__ - INFO - Done!