Hugging Face's logo

brthor
/
transcript-correction-loras

Safetensors
Model card Files
xet
 Community
brthor commited on
Commit
3d7271d
·
verified ·
1 Parent(s): d93c089

Delete meta-llama__meta-llama-3.1-8b/sft/checkpoints

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/README.md +0 -210
  2. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/adapter_config.json +0 -50
  3. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/adapter_model.safetensors +0 -3
  4. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/chat_template.jinja +0 -12
  5. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/optimizer.pt +0 -3
  6. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/rng_state.pth +0 -3
  7. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/scheduler.pt +0 -3
  8. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/tokenizer.json +0 -3
  9. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/tokenizer_config.json +0 -18
  10. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/trainer_state.json +0 -1615
  11. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/training_args.bin +0 -3
  12. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/README.md +0 -210
  13. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/adapter_config.json +0 -50
  14. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/adapter_model.safetensors +0 -3
  15. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/chat_template.jinja +0 -12
  16. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/optimizer.pt +0 -3
  17. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/rng_state.pth +0 -3
  18. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/scheduler.pt +0 -3
  19. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/tokenizer.json +0 -3
  20. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/tokenizer_config.json +0 -18
  21. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/trainer_state.json +0 -0
  22. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/training_args.bin +0 -3
  23. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/README.md +0 -210
  24. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/adapter_config.json +0 -50
  25. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/adapter_model.safetensors +0 -3
  26. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/chat_template.jinja +0 -12
  27. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/optimizer.pt +0 -3
  28. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/rng_state.pth +0 -3
  29. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/scheduler.pt +0 -3
  30. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/tokenizer.json +0 -3
  31. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/tokenizer_config.json +0 -18
  32. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/trainer_state.json +0 -2142
  33. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/training_args.bin +0 -3
  34. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/README.md +0 -210
  35. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/adapter_config.json +0 -50
  36. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/adapter_model.safetensors +0 -3
  37. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/chat_template.jinja +0 -12
  38. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/optimizer.pt +0 -3
  39. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/rng_state.pth +0 -3
  40. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/scheduler.pt +0 -3
  41. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/tokenizer.json +0 -3
  42. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/tokenizer_config.json +0 -18
  43. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/trainer_state.json +0 -2101
  44. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/training_args.bin +0 -3
  45. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/README.md +0 -210
  46. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/adapter_config.json +0 -50
  47. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/adapter_model.safetensors +0 -3
  48. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/chat_template.jinja +0 -12
  49. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/optimizer.pt +0 -3
  50. meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/rng_state.pth +0 -3
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/README.md DELETED
@@ -1,210 +0,0 @@
1
- ---
2
- base_model: unsloth/Meta-Llama-3.1-8B
3
- library_name: peft
4
- pipeline_tag: text-generation
5
- tags:
6
- - base_model:adapter:unsloth/Meta-Llama-3.1-8B
7
- - lora
8
- - sft
9
- - transformers
10
- - trl
11
- - unsloth
12
- ---
13
-
14
- # Model Card for Model ID
15
-
16
- <!-- Provide a quick summary of what the model is/does. -->
17
-
18
-
19
-
20
- ## Model Details
21
-
22
- ### Model Description
23
-
24
- <!-- Provide a longer summary of what this model is. -->
25
-
26
-
27
-
28
- - **Developed by:** [More Information Needed]
29
- - **Funded by [optional]:** [More Information Needed]
30
- - **Shared by [optional]:** [More Information Needed]
31
- - **Model type:** [More Information Needed]
32
- - **Language(s) (NLP):** [More Information Needed]
33
- - **License:** [More Information Needed]
34
- - **Finetuned from model [optional]:** [More Information Needed]
35
-
36
- ### Model Sources [optional]
37
-
38
- <!-- Provide the basic links for the model. -->
39
-
40
- - **Repository:** [More Information Needed]
41
- - **Paper [optional]:** [More Information Needed]
42
- - **Demo [optional]:** [More Information Needed]
43
-
44
- ## Uses
45
-
46
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
-
48
- ### Direct Use
49
-
50
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
-
52
- [More Information Needed]
53
-
54
- ### Downstream Use [optional]
55
-
56
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
-
58
- [More Information Needed]
59
-
60
- ### Out-of-Scope Use
61
-
62
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
-
64
- [More Information Needed]
65
-
66
- ## Bias, Risks, and Limitations
67
-
68
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
-
70
- [More Information Needed]
71
-
72
- ### Recommendations
73
-
74
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
-
76
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
-
78
- ## How to Get Started with the Model
79
-
80
- Use the code below to get started with the model.
81
-
82
- [More Information Needed]
83
-
84
- ## Training Details
85
-
86
- ### Training Data
87
-
88
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
-
90
- [More Information Needed]
91
-
92
- ### Training Procedure
93
-
94
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
-
96
- #### Preprocessing [optional]
97
-
98
- [More Information Needed]
99
-
100
-
101
- #### Training Hyperparameters
102
-
103
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
-
105
- #### Speeds, Sizes, Times [optional]
106
-
107
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
-
109
- [More Information Needed]
110
-
111
- ## Evaluation
112
-
113
- <!-- This section describes the evaluation protocols and provides the results. -->
114
-
115
- ### Testing Data, Factors & Metrics
116
-
117
- #### Testing Data
118
-
119
- <!-- This should link to a Dataset Card if possible. -->
120
-
121
- [More Information Needed]
122
-
123
- #### Factors
124
-
125
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
-
127
- [More Information Needed]
128
-
129
- #### Metrics
130
-
131
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
-
133
- [More Information Needed]
134
-
135
- ### Results
136
-
137
- [More Information Needed]
138
-
139
- #### Summary
140
-
141
-
142
-
143
- ## Model Examination [optional]
144
-
145
- <!-- Relevant interpretability work for the model goes here -->
146
-
147
- [More Information Needed]
148
-
149
- ## Environmental Impact
150
-
151
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
-
153
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
-
155
- - **Hardware Type:** [More Information Needed]
156
- - **Hours used:** [More Information Needed]
157
- - **Cloud Provider:** [More Information Needed]
158
- - **Compute Region:** [More Information Needed]
159
- - **Carbon Emitted:** [More Information Needed]
160
-
161
- ## Technical Specifications [optional]
162
-
163
- ### Model Architecture and Objective
164
-
165
- [More Information Needed]
166
-
167
- ### Compute Infrastructure
168
-
169
- [More Information Needed]
170
-
171
- #### Hardware
172
-
173
- [More Information Needed]
174
-
175
- #### Software
176
-
177
- [More Information Needed]
178
-
179
- ## Citation [optional]
180
-
181
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
-
183
- **BibTeX:**
184
-
185
- [More Information Needed]
186
-
187
- **APA:**
188
-
189
- [More Information Needed]
190
-
191
- ## Glossary [optional]
192
-
193
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
-
195
- [More Information Needed]
196
-
197
- ## More Information [optional]
198
-
199
- [More Information Needed]
200
-
201
- ## Model Card Authors [optional]
202
-
203
- [More Information Needed]
204
-
205
- ## Model Card Contact
206
-
207
- [More Information Needed]
208
- ### Framework versions
209
-
210
- - PEFT 0.18.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/adapter_config.json DELETED
@@ -1,50 +0,0 @@
1
- {
2
- "alora_invocation_tokens": null,
3
- "alpha_pattern": {},
4
- "arrow_config": null,
5
- "auto_mapping": {
6
- "base_model_class": "LlamaForCausalLM",
7
- "parent_library": "transformers.models.llama.modeling_llama",
8
- "unsloth_fixed": true
9
- },
10
- "base_model_name_or_path": "unsloth/Meta-Llama-3.1-8B",
11
- "bias": "none",
12
- "corda_config": null,
13
- "ensure_weight_tying": false,
14
- "eva_config": null,
15
- "exclude_modules": null,
16
- "fan_in_fan_out": false,
17
- "inference_mode": true,
18
- "init_lora_weights": true,
19
- "layer_replication": null,
20
- "layers_pattern": null,
21
- "layers_to_transform": null,
22
- "loftq_config": {},
23
- "lora_alpha": 16,
24
- "lora_bias": false,
25
- "lora_dropout": 0.0,
26
- "megatron_config": null,
27
- "megatron_core": "megatron.core",
28
- "modules_to_save": null,
29
- "peft_type": "LORA",
30
- "peft_version": "0.18.1",
31
- "qalora_group_size": 16,
32
- "r": 8,
33
- "rank_pattern": {},
34
- "revision": null,
35
- "target_modules": [
36
- "q_proj",
37
- "k_proj",
38
- "up_proj",
39
- "gate_proj",
40
- "down_proj",
41
- "o_proj",
42
- "v_proj"
43
- ],
44
- "target_parameters": null,
45
- "task_type": "CAUSAL_LM",
46
- "trainable_token_indices": null,
47
- "use_dora": false,
48
- "use_qalora": false,
49
- "use_rslora": false
50
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3998cb16f83a5deb3ff6fe0e36e9c165afe76105855779789564d6663ece011e
3
- size 83945296
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/chat_template.jinja DELETED
@@ -1,12 +0,0 @@
1
- {{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ messages[0]['content'] + '
2
-
3
- ' }}{% set loop_messages = messages[1:] %}{% else %}{{ '' + '
4
-
5
- ' }}{% set loop_messages = messages %}{% endif %}{% for message in loop_messages %}{% if message['role'] == 'user' %}{{ '### Instruction:
6
- ' + message['content'] + '
7
-
8
- ' }}{% elif message['role'] == 'assistant' %}{{ '### Response:
9
- ' + message['content'] + eos_token + '
10
-
11
- ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '### Response:
12
- ' }}{% endif %}
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:167a8807a4f09ae154cc13e8241ad219ccc02ef35bfecd8cf5da4005845ac15d
3
- size 43127973
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d569a9c96ff4e89d32dfe8ca0b97952cc2edcd76f89c14caa7e57238c4b46c8a
3
- size 14645
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:4812f164ea7757b51fbfb4d00b1fa6325746cb792d7c7ec0dd7e074068a93cfb
3
- size 1465
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/tokenizer.json DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
3
- size 17209920
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/tokenizer_config.json DELETED
@@ -1,18 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "bos_token": "<|begin_of_text|>",
4
- "clean_up_tokenization_spaces": true,
5
- "eos_token": "<|end_of_text|>",
6
- "from_slow": true,
7
- "is_local": false,
8
- "legacy": false,
9
- "model_input_names": [
10
- "input_ids",
11
- "attention_mask"
12
- ],
13
- "model_max_length": 131072,
14
- "pad_token": "<|finetune_right_pad_id|>",
15
- "padding_side": "right",
16
- "tokenizer_class": "TokenizersBackend",
17
- "unk_token": null
18
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/trainer_state.json DELETED
@@ -1,1615 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 0.7489768076398363,
6
- "eval_steps": 366,
7
- "global_step": 1098,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.0034106412005457027,
14
- "grad_norm": 3.1571648120880127,
15
- "learning_rate": 2.542372881355932e-06,
16
- "loss": 1.6981744766235352,
17
- "step": 5
18
- },
19
- {
20
- "epoch": 0.0068212824010914054,
21
- "grad_norm": 9.80073070526123,
22
- "learning_rate": 5.720338983050847e-06,
23
- "loss": 1.5801130294799806,
24
- "step": 10
25
- },
26
- {
27
- "epoch": 0.010231923601637109,
28
- "grad_norm": 4.481558799743652,
29
- "learning_rate": 8.898305084745763e-06,
30
- "loss": 2.1718717575073243,
31
- "step": 15
32
- },
33
- {
34
- "epoch": 0.013642564802182811,
35
- "grad_norm": 7.221553802490234,
36
- "learning_rate": 1.2076271186440677e-05,
37
- "loss": 1.5270480155944823,
38
- "step": 20
39
- },
40
- {
41
- "epoch": 0.017053206002728513,
42
- "grad_norm": 5.330269813537598,
43
- "learning_rate": 1.5254237288135592e-05,
44
- "loss": 0.9586630821228027,
45
- "step": 25
46
- },
47
- {
48
- "epoch": 0.020463847203274217,
49
- "grad_norm": 1.7404323816299438,
50
- "learning_rate": 1.8432203389830506e-05,
51
- "loss": 0.48505802154541017,
52
- "step": 30
53
- },
54
- {
55
- "epoch": 0.023874488403819918,
56
- "grad_norm": 1.2418887615203857,
57
- "learning_rate": 2.1610169491525424e-05,
58
- "loss": 0.24452033042907714,
59
- "step": 35
60
- },
61
- {
62
- "epoch": 0.027285129604365622,
63
- "grad_norm": 1.5928819179534912,
64
- "learning_rate": 2.4788135593220338e-05,
65
- "loss": 0.25337533950805663,
66
- "step": 40
67
- },
68
- {
69
- "epoch": 0.030695770804911322,
70
- "grad_norm": 2.0335769653320312,
71
- "learning_rate": 2.796610169491525e-05,
72
- "loss": 0.1938886046409607,
73
- "step": 45
74
- },
75
- {
76
- "epoch": 0.034106412005457026,
77
- "grad_norm": 0.18029411137104034,
78
- "learning_rate": 3.114406779661017e-05,
79
- "loss": 0.1834133505821228,
80
- "step": 50
81
- },
82
- {
83
- "epoch": 0.03751705320600273,
84
- "grad_norm": 1.7757192850112915,
85
- "learning_rate": 3.432203389830508e-05,
86
- "loss": 0.15151114463806153,
87
- "step": 55
88
- },
89
- {
90
- "epoch": 0.040927694406548434,
91
- "grad_norm": 2.1376893520355225,
92
- "learning_rate": 3.75e-05,
93
- "loss": 0.21634674072265625,
94
- "step": 60
95
- },
96
- {
97
- "epoch": 0.04433833560709413,
98
- "grad_norm": 2.196387767791748,
99
- "learning_rate": 4.067796610169491e-05,
100
- "loss": 0.11320395469665527,
101
- "step": 65
102
- },
103
- {
104
- "epoch": 0.047748976807639835,
105
- "grad_norm": 1.5888807773590088,
106
- "learning_rate": 4.3855932203389825e-05,
107
- "loss": 0.11050152778625488,
108
- "step": 70
109
- },
110
- {
111
- "epoch": 0.05115961800818554,
112
- "grad_norm": 4.606944561004639,
113
- "learning_rate": 4.703389830508474e-05,
114
- "loss": 0.23255281448364257,
115
- "step": 75
116
- },
117
- {
118
- "epoch": 0.054570259208731244,
119
- "grad_norm": 1.7308639287948608,
120
- "learning_rate": 5.021186440677966e-05,
121
- "loss": 0.08995866179466247,
122
- "step": 80
123
- },
124
- {
125
- "epoch": 0.05798090040927694,
126
- "grad_norm": 2.622438430786133,
127
- "learning_rate": 5.338983050847457e-05,
128
- "loss": 0.05715223550796509,
129
- "step": 85
130
- },
131
- {
132
- "epoch": 0.061391541609822645,
133
- "grad_norm": 0.34140461683273315,
134
- "learning_rate": 5.656779661016949e-05,
135
- "loss": 0.06703727245330811,
136
- "step": 90
137
- },
138
- {
139
- "epoch": 0.06480218281036836,
140
- "grad_norm": 2.7264959812164307,
141
- "learning_rate": 5.97457627118644e-05,
142
- "loss": 0.10092716217041016,
143
- "step": 95
144
- },
145
- {
146
- "epoch": 0.06821282401091405,
147
- "grad_norm": 2.0234780311584473,
148
- "learning_rate": 6.292372881355932e-05,
149
- "loss": 0.03579937815666199,
150
- "step": 100
151
- },
152
- {
153
- "epoch": 0.07162346521145975,
154
- "grad_norm": 1.3716133832931519,
155
- "learning_rate": 6.610169491525423e-05,
156
- "loss": 0.07851418852806091,
157
- "step": 105
158
- },
159
- {
160
- "epoch": 0.07503410641200546,
161
- "grad_norm": 1.2307227849960327,
162
- "learning_rate": 6.927966101694914e-05,
163
- "loss": 0.07370938658714295,
164
- "step": 110
165
- },
166
- {
167
- "epoch": 0.07844474761255116,
168
- "grad_norm": 1.7142690420150757,
169
- "learning_rate": 7.245762711864406e-05,
170
- "loss": 0.08064572811126709,
171
- "step": 115
172
- },
173
- {
174
- "epoch": 0.08185538881309687,
175
- "grad_norm": 1.343595266342163,
176
- "learning_rate": 7.499999439507554e-05,
177
- "loss": 0.11841402053833008,
178
- "step": 120
179
- },
180
- {
181
- "epoch": 0.08526603001364257,
182
- "grad_norm": 1.2511327266693115,
183
- "learning_rate": 7.499979822289558e-05,
184
- "loss": 0.06974860429763793,
185
- "step": 125
186
- },
187
- {
188
- "epoch": 0.08867667121418826,
189
- "grad_norm": 1.5642656087875366,
190
- "learning_rate": 7.49993218061684e-05,
191
- "loss": 0.10972714424133301,
192
- "step": 130
193
- },
194
- {
195
- "epoch": 0.09208731241473397,
196
- "grad_norm": 1.6669789552688599,
197
- "learning_rate": 7.499856514845436e-05,
198
- "loss": 0.09864612817764282,
199
- "step": 135
200
- },
201
- {
202
- "epoch": 0.09549795361527967,
203
- "grad_norm": 0.9789333343505859,
204
- "learning_rate": 7.499752825540815e-05,
205
- "loss": 0.0699621558189392,
206
- "step": 140
207
- },
208
- {
209
- "epoch": 0.09890859481582538,
210
- "grad_norm": 0.41792476177215576,
211
- "learning_rate": 7.499621113477873e-05,
212
- "loss": 0.06734349727630615,
213
- "step": 145
214
- },
215
- {
216
- "epoch": 0.10231923601637108,
217
- "grad_norm": 1.5968533754348755,
218
- "learning_rate": 7.499461379640919e-05,
219
- "loss": 0.060729533433914185,
220
- "step": 150
221
- },
222
- {
223
- "epoch": 0.10572987721691678,
224
- "grad_norm": 0.8512193560600281,
225
- "learning_rate": 7.499273625223683e-05,
226
- "loss": 0.052730172872543335,
227
- "step": 155
228
- },
229
- {
230
- "epoch": 0.10914051841746249,
231
- "grad_norm": 1.3257211446762085,
232
- "learning_rate": 7.499057851629299e-05,
233
- "loss": 0.10094538927078248,
234
- "step": 160
235
- },
236
- {
237
- "epoch": 0.11255115961800818,
238
- "grad_norm": 0.659118115901947,
239
- "learning_rate": 7.498814060470288e-05,
240
- "loss": 0.01604635864496231,
241
- "step": 165
242
- },
243
- {
244
- "epoch": 0.11596180081855388,
245
- "grad_norm": 2.454979181289673,
246
- "learning_rate": 7.49854225356856e-05,
247
- "loss": 0.13272385597229003,
248
- "step": 170
249
- },
250
- {
251
- "epoch": 0.11937244201909959,
252
- "grad_norm": 2.510040521621704,
253
- "learning_rate": 7.498242432955388e-05,
254
- "loss": 0.08396227955818177,
255
- "step": 175
256
- },
257
- {
258
- "epoch": 0.12278308321964529,
259
- "grad_norm": 0.2637259364128113,
260
- "learning_rate": 7.4979146008714e-05,
261
- "loss": 0.05852065086364746,
262
- "step": 180
263
- },
264
- {
265
- "epoch": 0.126193724420191,
266
- "grad_norm": 1.4320851564407349,
267
- "learning_rate": 7.497558759766564e-05,
268
- "loss": 0.10392802953720093,
269
- "step": 185
270
- },
271
- {
272
- "epoch": 0.1296043656207367,
273
- "grad_norm": 1.273027777671814,
274
- "learning_rate": 7.497174912300156e-05,
275
- "loss": 0.08062989711761474,
276
- "step": 190
277
- },
278
- {
279
- "epoch": 0.1330150068212824,
280
- "grad_norm": 0.8102871179580688,
281
- "learning_rate": 7.496763061340759e-05,
282
- "loss": 0.07294153571128845,
283
- "step": 195
284
- },
285
- {
286
- "epoch": 0.1364256480218281,
287
- "grad_norm": 1.951328992843628,
288
- "learning_rate": 7.496323209966228e-05,
289
- "loss": 0.07738351821899414,
290
- "step": 200
291
- },
292
- {
293
- "epoch": 0.13983628922237382,
294
- "grad_norm": 0.3880983889102936,
295
- "learning_rate": 7.495855361463674e-05,
296
- "loss": 0.07225048542022705,
297
- "step": 205
298
- },
299
- {
300
- "epoch": 0.1432469304229195,
301
- "grad_norm": 3.3205058574676514,
302
- "learning_rate": 7.495359519329433e-05,
303
- "loss": 0.05682974457740784,
304
- "step": 210
305
- },
306
- {
307
- "epoch": 0.1466575716234652,
308
- "grad_norm": 0.9203559160232544,
309
- "learning_rate": 7.49483568726905e-05,
310
- "loss": 0.08767472505569458,
311
- "step": 215
312
- },
313
- {
314
- "epoch": 0.15006821282401092,
315
- "grad_norm": 0.585360586643219,
316
- "learning_rate": 7.494283869197239e-05,
317
- "loss": 0.039227068424224854,
318
- "step": 220
319
- },
320
- {
321
- "epoch": 0.1534788540245566,
322
- "grad_norm": 1.7096059322357178,
323
- "learning_rate": 7.493704069237862e-05,
324
- "loss": 0.10281096696853638,
325
- "step": 225
326
- },
327
- {
328
- "epoch": 0.15688949522510232,
329
- "grad_norm": 0.4110204875469208,
330
- "learning_rate": 7.493096291723898e-05,
331
- "loss": 0.04346161186695099,
332
- "step": 230
333
- },
334
- {
335
- "epoch": 0.16030013642564803,
336
- "grad_norm": 1.3272292613983154,
337
- "learning_rate": 7.492460541197404e-05,
338
- "loss": 0.049719154834747314,
339
- "step": 235
340
- },
341
- {
342
- "epoch": 0.16371077762619374,
343
- "grad_norm": 1.1005016565322876,
344
- "learning_rate": 7.491796822409494e-05,
345
- "loss": 0.09335108399391175,
346
- "step": 240
347
- },
348
- {
349
- "epoch": 0.16712141882673942,
350
- "grad_norm": 0.7811501026153564,
351
- "learning_rate": 7.491105140320285e-05,
352
- "loss": 0.05943926572799683,
353
- "step": 245
354
- },
355
- {
356
- "epoch": 0.17053206002728513,
357
- "grad_norm": 1.4607417583465576,
358
- "learning_rate": 7.490385500098879e-05,
359
- "loss": 0.04385361075401306,
360
- "step": 250
361
- },
362
- {
363
- "epoch": 0.17394270122783084,
364
- "grad_norm": 0.394960880279541,
365
- "learning_rate": 7.489637907123308e-05,
366
- "loss": 0.04446137547492981,
367
- "step": 255
368
- },
369
- {
370
- "epoch": 0.17735334242837653,
371
- "grad_norm": 0.8768635988235474,
372
- "learning_rate": 7.488862366980505e-05,
373
- "loss": 0.04143576025962829,
374
- "step": 260
375
- },
376
- {
377
- "epoch": 0.18076398362892224,
378
- "grad_norm": 1.9996010065078735,
379
- "learning_rate": 7.488058885466262e-05,
380
- "loss": 0.07952215671539306,
381
- "step": 265
382
- },
383
- {
384
- "epoch": 0.18417462482946795,
385
- "grad_norm": 0.03770223259925842,
386
- "learning_rate": 7.487227468585178e-05,
387
- "loss": 0.02531362771987915,
388
- "step": 270
389
- },
390
- {
391
- "epoch": 0.18758526603001363,
392
- "grad_norm": 0.26082542538642883,
393
- "learning_rate": 7.486368122550619e-05,
394
- "loss": 0.09930967688560485,
395
- "step": 275
396
- },
397
- {
398
- "epoch": 0.19099590723055934,
399
- "grad_norm": 5.622270584106445,
400
- "learning_rate": 7.485480853784677e-05,
401
- "loss": 0.06534865498542786,
402
- "step": 280
403
- },
404
- {
405
- "epoch": 0.19440654843110505,
406
- "grad_norm": 0.5298851132392883,
407
- "learning_rate": 7.484565668918111e-05,
408
- "loss": 0.06109699010848999,
409
- "step": 285
410
- },
411
- {
412
- "epoch": 0.19781718963165076,
413
- "grad_norm": 1.4887421131134033,
414
- "learning_rate": 7.483622574790308e-05,
415
- "loss": 0.048966211080551145,
416
- "step": 290
417
- },
418
- {
419
- "epoch": 0.20122783083219645,
420
- "grad_norm": 0.5699282884597778,
421
- "learning_rate": 7.482651578449223e-05,
422
- "loss": 0.05427658557891846,
423
- "step": 295
424
- },
425
- {
426
- "epoch": 0.20463847203274216,
427
- "grad_norm": 1.6645292043685913,
428
- "learning_rate": 7.481652687151339e-05,
429
- "loss": 0.037466832995414735,
430
- "step": 300
431
- },
432
- {
433
- "epoch": 0.20804911323328787,
434
- "grad_norm": 0.4979431629180908,
435
- "learning_rate": 7.480625908361593e-05,
436
- "loss": 0.019084173440933227,
437
- "step": 305
438
- },
439
- {
440
- "epoch": 0.21145975443383355,
441
- "grad_norm": 2.73081636428833,
442
- "learning_rate": 7.479571249753339e-05,
443
- "loss": 0.07597044706344605,
444
- "step": 310
445
- },
446
- {
447
- "epoch": 0.21487039563437926,
448
- "grad_norm": 0.009097559377551079,
449
- "learning_rate": 7.478488719208281e-05,
450
- "loss": 0.017771795392036438,
451
- "step": 315
452
- },
453
- {
454
- "epoch": 0.21828103683492497,
455
- "grad_norm": 1.5284112691879272,
456
- "learning_rate": 7.477378324816419e-05,
457
- "loss": 0.07524526119232178,
458
- "step": 320
459
- },
460
- {
461
- "epoch": 0.22169167803547066,
462
- "grad_norm": 1.400959849357605,
463
- "learning_rate": 7.47624007487598e-05,
464
- "loss": 0.0357323557138443,
465
- "step": 325
466
- },
467
- {
468
- "epoch": 0.22510231923601637,
469
- "grad_norm": 0.5988397598266602,
470
- "learning_rate": 7.47507397789337e-05,
471
- "loss": 0.06072888970375061,
472
- "step": 330
473
- },
474
- {
475
- "epoch": 0.22851296043656208,
476
- "grad_norm": 0.18309183418750763,
477
- "learning_rate": 7.473880042583092e-05,
478
- "loss": 0.03904334008693695,
479
- "step": 335
480
- },
481
- {
482
- "epoch": 0.23192360163710776,
483
- "grad_norm": 0.7360084056854248,
484
- "learning_rate": 7.472658277867702e-05,
485
- "loss": 0.05045387148857117,
486
- "step": 340
487
- },
488
- {
489
- "epoch": 0.23533424283765347,
490
- "grad_norm": 2.315072536468506,
491
- "learning_rate": 7.471408692877724e-05,
492
- "loss": 0.07920202016830444,
493
- "step": 345
494
- },
495
- {
496
- "epoch": 0.23874488403819918,
497
- "grad_norm": 1.2811086177825928,
498
- "learning_rate": 7.470131296951592e-05,
499
- "loss": 0.05552580952644348,
500
- "step": 350
501
- },
502
- {
503
- "epoch": 0.2421555252387449,
504
- "grad_norm": 4.006563186645508,
505
- "learning_rate": 7.468826099635578e-05,
506
- "loss": 0.1419215679168701,
507
- "step": 355
508
- },
509
- {
510
- "epoch": 0.24556616643929058,
511
- "grad_norm": 1.1540688276290894,
512
- "learning_rate": 7.467493110683718e-05,
513
- "loss": 0.03980849981307984,
514
- "step": 360
515
- },
516
- {
517
- "epoch": 0.2489768076398363,
518
- "grad_norm": 1.5472272634506226,
519
- "learning_rate": 7.466132340057742e-05,
520
- "loss": 0.020862475037574768,
521
- "step": 365
522
- },
523
- {
524
- "epoch": 0.24965893587994542,
525
- "eval_loss": 0.11654457449913025,
526
- "eval_runtime": 1.0333,
527
- "eval_samples_per_second": 72.584,
528
- "eval_steps_per_second": 1.936,
529
- "step": 366
530
- },
531
- {
532
- "eval_cer_subset": 0.05232320479629377,
533
- "eval_cer_subset_edit_distance": 384,
534
- "eval_cer_subset_groups": 75,
535
- "eval_cer_subset_items": 75,
536
- "eval_cer_subset_ref_chars": 7339,
537
- "step": 366
538
- },
539
- {
540
- "epoch": 0.252387448840382,
541
- "grad_norm": 1.730627417564392,
542
- "learning_rate": 7.464743797927002e-05,
543
- "loss": 0.11239330768585205,
544
- "step": 370
545
- },
546
- {
547
- "epoch": 0.2557980900409277,
548
- "grad_norm": 0.1921682506799698,
549
- "learning_rate": 7.463327494668388e-05,
550
- "loss": 0.09260941743850708,
551
- "step": 375
552
- },
553
- {
554
- "epoch": 0.2592087312414734,
555
- "grad_norm": 1.9259151220321655,
556
- "learning_rate": 7.461883440866259e-05,
557
- "loss": 0.03999299705028534,
558
- "step": 380
559
- },
560
- {
561
- "epoch": 0.2626193724420191,
562
- "grad_norm": 0.0488249845802784,
563
- "learning_rate": 7.460411647312358e-05,
564
- "loss": 0.01459498256444931,
565
- "step": 385
566
- },
567
- {
568
- "epoch": 0.2660300136425648,
569
- "grad_norm": 1.1967735290527344,
570
- "learning_rate": 7.458912125005732e-05,
571
- "loss": 0.17716412544250487,
572
- "step": 390
573
- },
574
- {
575
- "epoch": 0.2694406548431105,
576
- "grad_norm": 1.0083205699920654,
577
- "learning_rate": 7.457384885152655e-05,
578
- "loss": 0.08738511800765991,
579
- "step": 395
580
- },
581
- {
582
- "epoch": 0.2728512960436562,
583
- "grad_norm": 0.6705593466758728,
584
- "learning_rate": 7.455829939166539e-05,
585
- "loss": 0.026945650577545166,
586
- "step": 400
587
- },
588
- {
589
- "epoch": 0.2762619372442019,
590
- "grad_norm": 1.0791361331939697,
591
- "learning_rate": 7.45424729866785e-05,
592
- "loss": 0.03804046213626862,
593
- "step": 405
594
- },
595
- {
596
- "epoch": 0.27967257844474763,
597
- "grad_norm": 0.7377910017967224,
598
- "learning_rate": 7.452636975484021e-05,
599
- "loss": 0.0464675635099411,
600
- "step": 410
601
- },
602
- {
603
- "epoch": 0.2830832196452933,
604
- "grad_norm": 0.8061625957489014,
605
- "learning_rate": 7.450998981649365e-05,
606
- "loss": 0.02737331986427307,
607
- "step": 415
608
- },
609
- {
610
- "epoch": 0.286493860845839,
611
- "grad_norm": 0.2580685019493103,
612
- "learning_rate": 7.449333329404982e-05,
613
- "loss": 0.018785761296749116,
614
- "step": 420
615
- },
616
- {
617
- "epoch": 0.28990450204638474,
618
- "grad_norm": 0.017052194103598595,
619
- "learning_rate": 7.447640031198675e-05,
620
- "loss": 0.11424320936203003,
621
- "step": 425
622
- },
623
- {
624
- "epoch": 0.2933151432469304,
625
- "grad_norm": 0.2855948805809021,
626
- "learning_rate": 7.445919099684845e-05,
627
- "loss": 0.012821969389915467,
628
- "step": 430
629
- },
630
- {
631
- "epoch": 0.2967257844474761,
632
- "grad_norm": 1.5070558786392212,
633
- "learning_rate": 7.444170547724405e-05,
634
- "loss": 0.037783479690551756,
635
- "step": 435
636
- },
637
- {
638
- "epoch": 0.30013642564802184,
639
- "grad_norm": 0.18241967260837555,
640
- "learning_rate": 7.442394388384684e-05,
641
- "loss": 0.03347713351249695,
642
- "step": 440
643
- },
644
- {
645
- "epoch": 0.3035470668485675,
646
- "grad_norm": 0.37319424748420715,
647
- "learning_rate": 7.440590634939327e-05,
648
- "loss": 0.05499382615089417,
649
- "step": 445
650
- },
651
- {
652
- "epoch": 0.3069577080491132,
653
- "grad_norm": 0.11083097755908966,
654
- "learning_rate": 7.438759300868193e-05,
655
- "loss": 0.021977408230304717,
656
- "step": 450
657
- },
658
- {
659
- "epoch": 0.31036834924965895,
660
- "grad_norm": 0.24985608458518982,
661
- "learning_rate": 7.436900399857261e-05,
662
- "loss": 0.07826730608940125,
663
- "step": 455
664
- },
665
- {
666
- "epoch": 0.31377899045020463,
667
- "grad_norm": 2.5360186100006104,
668
- "learning_rate": 7.43501394579852e-05,
669
- "loss": 0.080865478515625,
670
- "step": 460
671
- },
672
- {
673
- "epoch": 0.3171896316507503,
674
- "grad_norm": 0.7053658366203308,
675
- "learning_rate": 7.433099952789876e-05,
676
- "loss": 0.012464526295661926,
677
- "step": 465
678
- },
679
- {
680
- "epoch": 0.32060027285129605,
681
- "grad_norm": 0.3297032117843628,
682
- "learning_rate": 7.43115843513503e-05,
683
- "loss": 0.05623521208763123,
684
- "step": 470
685
- },
686
- {
687
- "epoch": 0.32401091405184174,
688
- "grad_norm": 1.3899471759796143,
689
- "learning_rate": 7.42918940734339e-05,
690
- "loss": 0.07306370735168458,
691
- "step": 475
692
- },
693
- {
694
- "epoch": 0.3274215552523875,
695
- "grad_norm": 0.9437380433082581,
696
- "learning_rate": 7.427192884129948e-05,
697
- "loss": 0.058290761709213254,
698
- "step": 480
699
- },
700
- {
701
- "epoch": 0.33083219645293316,
702
- "grad_norm": 1.8157323598861694,
703
- "learning_rate": 7.42516888041518e-05,
704
- "loss": 0.058532989025115965,
705
- "step": 485
706
- },
707
- {
708
- "epoch": 0.33424283765347884,
709
- "grad_norm": 0.6275774836540222,
710
- "learning_rate": 7.423117411324924e-05,
711
- "loss": 0.0624964714050293,
712
- "step": 490
713
- },
714
- {
715
- "epoch": 0.3376534788540246,
716
- "grad_norm": 0.6674565672874451,
717
- "learning_rate": 7.421038492190278e-05,
718
- "loss": 0.020014175772666933,
719
- "step": 495
720
- },
721
- {
722
- "epoch": 0.34106412005457026,
723
- "grad_norm": 0.6229842901229858,
724
- "learning_rate": 7.418932138547481e-05,
725
- "loss": 0.03286575376987457,
726
- "step": 500
727
- },
728
- {
729
- "epoch": 0.34447476125511595,
730
- "grad_norm": 1.441677212715149,
731
- "learning_rate": 7.41679836613779e-05,
732
- "loss": 0.04557921886444092,
733
- "step": 505
734
- },
735
- {
736
- "epoch": 0.3478854024556617,
737
- "grad_norm": 0.8439592719078064,
738
- "learning_rate": 7.414637190907379e-05,
739
- "loss": 0.027792316675186158,
740
- "step": 510
741
- },
742
- {
743
- "epoch": 0.35129604365620737,
744
- "grad_norm": 0.1357346475124359,
745
- "learning_rate": 7.412448629007198e-05,
746
- "loss": 0.024153730273246764,
747
- "step": 515
748
- },
749
- {
750
- "epoch": 0.35470668485675305,
751
- "grad_norm": 0.11876281350851059,
752
- "learning_rate": 7.41023269679287e-05,
753
- "loss": 0.11651531457901002,
754
- "step": 520
755
- },
756
- {
757
- "epoch": 0.3581173260572988,
758
- "grad_norm": 0.8576210737228394,
759
- "learning_rate": 7.407989410824566e-05,
760
- "loss": 0.045156928896903994,
761
- "step": 525
762
- },
763
- {
764
- "epoch": 0.3615279672578445,
765
- "grad_norm": 0.39947113394737244,
766
- "learning_rate": 7.40571878786687e-05,
767
- "loss": 0.02562606930732727,
768
- "step": 530
769
- },
770
- {
771
- "epoch": 0.36493860845839016,
772
- "grad_norm": 0.8822716474533081,
773
- "learning_rate": 7.403420844888668e-05,
774
- "loss": 0.05394383668899536,
775
- "step": 535
776
- },
777
- {
778
- "epoch": 0.3683492496589359,
779
- "grad_norm": 1.8026832342147827,
780
- "learning_rate": 7.40109559906301e-05,
781
- "loss": 0.07706952095031738,
782
- "step": 540
783
- },
784
- {
785
- "epoch": 0.3717598908594816,
786
- "grad_norm": 0.28902706503868103,
787
- "learning_rate": 7.398743067766987e-05,
788
- "loss": 0.0352792352437973,
789
- "step": 545
790
- },
791
- {
792
- "epoch": 0.37517053206002726,
793
- "grad_norm": 0.2759908437728882,
794
- "learning_rate": 7.396363268581609e-05,
795
- "loss": 0.038266700506210324,
796
- "step": 550
797
- },
798
- {
799
- "epoch": 0.378581173260573,
800
- "grad_norm": 1.0520153045654297,
801
- "learning_rate": 7.39395621929165e-05,
802
- "loss": 0.04206843376159668,
803
- "step": 555
804
- },
805
- {
806
- "epoch": 0.3819918144611187,
807
- "grad_norm": 0.29290419816970825,
808
- "learning_rate": 7.391521937885543e-05,
809
- "loss": 0.04060278534889221,
810
- "step": 560
811
- },
812
- {
813
- "epoch": 0.38540245566166437,
814
- "grad_norm": 0.11243477463722229,
815
- "learning_rate": 7.389060442555228e-05,
816
- "loss": 0.05412468910217285,
817
- "step": 565
818
- },
819
- {
820
- "epoch": 0.3888130968622101,
821
- "grad_norm": 1.9416879415512085,
822
- "learning_rate": 7.386571751696019e-05,
823
- "loss": 0.02231921851634979,
824
- "step": 570
825
- },
826
- {
827
- "epoch": 0.3922237380627558,
828
- "grad_norm": 0.5937671661376953,
829
- "learning_rate": 7.384055883906474e-05,
830
- "loss": 0.032561862468719484,
831
- "step": 575
832
- },
833
- {
834
- "epoch": 0.3956343792633015,
835
- "grad_norm": 0.026148535311222076,
836
- "learning_rate": 7.381512857988244e-05,
837
- "loss": 0.07647547125816345,
838
- "step": 580
839
- },
840
- {
841
- "epoch": 0.3990450204638472,
842
- "grad_norm": 0.7875772714614868,
843
- "learning_rate": 7.378942692945944e-05,
844
- "loss": 0.031203645467758178,
845
- "step": 585
846
- },
847
- {
848
- "epoch": 0.4024556616643929,
849
- "grad_norm": 0.45512810349464417,
850
- "learning_rate": 7.376345407987002e-05,
851
- "loss": 0.04238590002059937,
852
- "step": 590
853
- },
854
- {
855
- "epoch": 0.40586630286493863,
856
- "grad_norm": 1.66355299949646,
857
- "learning_rate": 7.373721022521521e-05,
858
- "loss": 0.052533066272735594,
859
- "step": 595
860
- },
861
- {
862
- "epoch": 0.4092769440654843,
863
- "grad_norm": 0.08107655495405197,
864
- "learning_rate": 7.371069556162133e-05,
865
- "loss": 0.017715978622436523,
866
- "step": 600
867
- },
868
- {
869
- "epoch": 0.41268758526603,
870
- "grad_norm": 0.32274800539016724,
871
- "learning_rate": 7.368391028723851e-05,
872
- "loss": 0.1379294991493225,
873
- "step": 605
874
- },
875
- {
876
- "epoch": 0.41609822646657574,
877
- "grad_norm": 1.8197475671768188,
878
- "learning_rate": 7.365685460223922e-05,
879
- "loss": 0.03312918543815613,
880
- "step": 610
881
- },
882
- {
883
- "epoch": 0.4195088676671214,
884
- "grad_norm": 0.1390945166349411,
885
- "learning_rate": 7.362952870881677e-05,
886
- "loss": 0.027584537863731384,
887
- "step": 615
888
- },
889
- {
890
- "epoch": 0.4229195088676671,
891
- "grad_norm": 0.9081276655197144,
892
- "learning_rate": 7.360193281118378e-05,
893
- "loss": 0.06143233776092529,
894
- "step": 620
895
- },
896
- {
897
- "epoch": 0.42633015006821284,
898
- "grad_norm": 0.07777975499629974,
899
- "learning_rate": 7.35740671155707e-05,
900
- "loss": 0.053598570823669436,
901
- "step": 625
902
- },
903
- {
904
- "epoch": 0.4297407912687585,
905
- "grad_norm": 0.9314269423484802,
906
- "learning_rate": 7.354593183022422e-05,
907
- "loss": 0.05946495532989502,
908
- "step": 630
909
- },
910
- {
911
- "epoch": 0.4331514324693042,
912
- "grad_norm": 0.5312000513076782,
913
- "learning_rate": 7.351752716540575e-05,
914
- "loss": 0.030707958340644836,
915
- "step": 635
916
- },
917
- {
918
- "epoch": 0.43656207366984995,
919
- "grad_norm": 0.855117917060852,
920
- "learning_rate": 7.348885333338984e-05,
921
- "loss": 0.09321808815002441,
922
- "step": 640
923
- },
924
- {
925
- "epoch": 0.43997271487039563,
926
- "grad_norm": 0.12914253771305084,
927
- "learning_rate": 7.345991054846257e-05,
928
- "loss": 0.010356919467449188,
929
- "step": 645
930
- },
931
- {
932
- "epoch": 0.4433833560709413,
933
- "grad_norm": 0.4129096567630768,
934
- "learning_rate": 7.343069902691999e-05,
935
- "loss": 0.054264682531356814,
936
- "step": 650
937
- },
938
- {
939
- "epoch": 0.44679399727148705,
940
- "grad_norm": 1.8499324321746826,
941
- "learning_rate": 7.340121898706643e-05,
942
- "loss": 0.050659948587417604,
943
- "step": 655
944
- },
945
- {
946
- "epoch": 0.45020463847203274,
947
- "grad_norm": 0.5490806698799133,
948
- "learning_rate": 7.337147064921299e-05,
949
- "loss": 0.07158003449440002,
950
- "step": 660
951
- },
952
- {
953
- "epoch": 0.4536152796725784,
954
- "grad_norm": 1.1408376693725586,
955
- "learning_rate": 7.334145423567575e-05,
956
- "loss": 0.08845412135124206,
957
- "step": 665
958
- },
959
- {
960
- "epoch": 0.45702592087312416,
961
- "grad_norm": 1.5242546796798706,
962
- "learning_rate": 7.331116997077426e-05,
963
- "loss": 0.07773985266685486,
964
- "step": 670
965
- },
966
- {
967
- "epoch": 0.46043656207366984,
968
- "grad_norm": 0.7061560153961182,
969
- "learning_rate": 7.32806180808297e-05,
970
- "loss": 0.047228410840034485,
971
- "step": 675
972
- },
973
- {
974
- "epoch": 0.4638472032742155,
975
- "grad_norm": 0.8088539838790894,
976
- "learning_rate": 7.324979879416333e-05,
977
- "loss": 0.03726888597011566,
978
- "step": 680
979
- },
980
- {
981
- "epoch": 0.46725784447476126,
982
- "grad_norm": 0.5670620799064636,
983
- "learning_rate": 7.321871234109472e-05,
984
- "loss": 0.02899191677570343,
985
- "step": 685
986
- },
987
- {
988
- "epoch": 0.47066848567530695,
989
- "grad_norm": 2.3821427822113037,
990
- "learning_rate": 7.318735895394e-05,
991
- "loss": 0.033483856916427614,
992
- "step": 690
993
- },
994
- {
995
- "epoch": 0.4740791268758527,
996
- "grad_norm": 0.8073883652687073,
997
- "learning_rate": 7.315573886701023e-05,
998
- "loss": 0.05756385326385498,
999
- "step": 695
1000
- },
1001
- {
1002
- "epoch": 0.47748976807639837,
1003
- "grad_norm": 0.09120920300483704,
1004
- "learning_rate": 7.31238523166095e-05,
1005
- "loss": 0.0338085800409317,
1006
- "step": 700
1007
- },
1008
- {
1009
- "epoch": 0.48090040927694405,
1010
- "grad_norm": 0.33443862199783325,
1011
- "learning_rate": 7.309169954103326e-05,
1012
- "loss": 0.00844155102968216,
1013
- "step": 705
1014
- },
1015
- {
1016
- "epoch": 0.4843110504774898,
1017
- "grad_norm": 0.4880702793598175,
1018
- "learning_rate": 7.305928078056657e-05,
1019
- "loss": 0.09532383680343628,
1020
- "step": 710
1021
- },
1022
- {
1023
- "epoch": 0.4877216916780355,
1024
- "grad_norm": 0.11862733215093613,
1025
- "learning_rate": 7.302659627748221e-05,
1026
- "loss": 0.01845739334821701,
1027
- "step": 715
1028
- },
1029
- {
1030
- "epoch": 0.49113233287858116,
1031
- "grad_norm": 0.03655651956796646,
1032
- "learning_rate": 7.299364627603892e-05,
1033
- "loss": 0.030477851629257202,
1034
- "step": 720
1035
- },
1036
- {
1037
- "epoch": 0.4945429740791269,
1038
- "grad_norm": 1.481441617012024,
1039
- "learning_rate": 7.29604310224796e-05,
1040
- "loss": 0.07586092352867127,
1041
- "step": 725
1042
- },
1043
- {
1044
- "epoch": 0.4979536152796726,
1045
- "grad_norm": 0.8510580658912659,
1046
- "learning_rate": 7.292695076502938e-05,
1047
- "loss": 0.03589251637458801,
1048
- "step": 730
1049
- },
1050
- {
1051
- "epoch": 0.49931787175989084,
1052
- "eval_loss": 0.061700768768787384,
1053
- "eval_runtime": 0.8886,
1054
- "eval_samples_per_second": 84.399,
1055
- "eval_steps_per_second": 2.251,
1056
- "step": 732
1057
- },
1058
- {
1059
- "eval_cer_subset": 0.021256301948494344,
1060
- "eval_cer_subset_edit_distance": 156,
1061
- "eval_cer_subset_groups": 75,
1062
- "eval_cer_subset_items": 75,
1063
- "eval_cer_subset_ref_chars": 7339,
1064
- "step": 732
1065
- },
1066
- {
1067
- "epoch": 0.5013642564802183,
1068
- "grad_norm": 0.49610504508018494,
1069
- "learning_rate": 7.28932057538939e-05,
1070
- "loss": 0.06440846920013428,
1071
- "step": 735
1072
- },
1073
- {
1074
- "epoch": 0.504774897680764,
1075
- "grad_norm": 0.5659550428390503,
1076
- "learning_rate": 7.285919624125732e-05,
1077
- "loss": 0.08426347374916077,
1078
- "step": 740
1079
- },
1080
- {
1081
- "epoch": 0.5081855388813097,
1082
- "grad_norm": 0.04723689705133438,
1083
- "learning_rate": 7.282492248128047e-05,
1084
- "loss": 0.07788341641426086,
1085
- "step": 745
1086
- },
1087
- {
1088
- "epoch": 0.5115961800818554,
1089
- "grad_norm": 0.720941960811615,
1090
- "learning_rate": 7.2790384730099e-05,
1091
- "loss": 0.03100808262825012,
1092
- "step": 750
1093
- },
1094
- {
1095
- "epoch": 0.515006821282401,
1096
- "grad_norm": 0.652988851070404,
1097
- "learning_rate": 7.275558324582138e-05,
1098
- "loss": 0.03954651951789856,
1099
- "step": 755
1100
- },
1101
- {
1102
- "epoch": 0.5184174624829468,
1103
- "grad_norm": 1.2214330434799194,
1104
- "learning_rate": 7.272051828852705e-05,
1105
- "loss": 0.019992084801197053,
1106
- "step": 760
1107
- },
1108
- {
1109
- "epoch": 0.5218281036834925,
1110
- "grad_norm": 1.292953372001648,
1111
- "learning_rate": 7.268519012026443e-05,
1112
- "loss": 0.07394988536834717,
1113
- "step": 765
1114
- },
1115
- {
1116
- "epoch": 0.5252387448840382,
1117
- "grad_norm": 1.1823278665542603,
1118
- "learning_rate": 7.264959900504901e-05,
1119
- "loss": 0.037449967861175534,
1120
- "step": 770
1121
- },
1122
- {
1123
- "epoch": 0.5286493860845839,
1124
- "grad_norm": 1.1314970254898071,
1125
- "learning_rate": 7.261374520886128e-05,
1126
- "loss": 0.04381995797157288,
1127
- "step": 775
1128
- },
1129
- {
1130
- "epoch": 0.5320600272851296,
1131
- "grad_norm": 0.02543286792933941,
1132
- "learning_rate": 7.257762899964486e-05,
1133
- "loss": 0.07130052447319031,
1134
- "step": 780
1135
- },
1136
- {
1137
- "epoch": 0.5354706684856753,
1138
- "grad_norm": 0.37440457940101624,
1139
- "learning_rate": 7.25412506473044e-05,
1140
- "loss": 0.050841158628463744,
1141
- "step": 785
1142
- },
1143
- {
1144
- "epoch": 0.538881309686221,
1145
- "grad_norm": 0.2532084882259369,
1146
- "learning_rate": 7.250461042370365e-05,
1147
- "loss": 0.03486245274543762,
1148
- "step": 790
1149
- },
1150
- {
1151
- "epoch": 0.5422919508867667,
1152
- "grad_norm": 1.0150209665298462,
1153
- "learning_rate": 7.246770860266333e-05,
1154
- "loss": 0.050749993324279784,
1155
- "step": 795
1156
- },
1157
- {
1158
- "epoch": 0.5457025920873124,
1159
- "grad_norm": 1.108716607093811,
1160
- "learning_rate": 7.24305454599592e-05,
1161
- "loss": 0.03166365325450897,
1162
- "step": 800
1163
- },
1164
- {
1165
- "epoch": 0.5491132332878581,
1166
- "grad_norm": 0.16657976806163788,
1167
- "learning_rate": 7.239312127331989e-05,
1168
- "loss": 0.05016656517982483,
1169
- "step": 805
1170
- },
1171
- {
1172
- "epoch": 0.5525238744884038,
1173
- "grad_norm": 0.005627671722322702,
1174
- "learning_rate": 7.235543632242488e-05,
1175
- "loss": 0.021701858937740327,
1176
- "step": 810
1177
- },
1178
- {
1179
- "epoch": 0.5559345156889495,
1180
- "grad_norm": 1.8796381950378418,
1181
- "learning_rate": 7.231749088890241e-05,
1182
- "loss": 0.061094462871551514,
1183
- "step": 815
1184
- },
1185
- {
1186
- "epoch": 0.5593451568894953,
1187
- "grad_norm": 0.020010950043797493,
1188
- "learning_rate": 7.227928525632737e-05,
1189
- "loss": 0.0238655224442482,
1190
- "step": 820
1191
- },
1192
- {
1193
- "epoch": 0.562755798090041,
1194
- "grad_norm": 1.9777793884277344,
1195
- "learning_rate": 7.224081971021914e-05,
1196
- "loss": 0.041665592789649965,
1197
- "step": 825
1198
- },
1199
- {
1200
- "epoch": 0.5661664392905866,
1201
- "grad_norm": 0.06644955277442932,
1202
- "learning_rate": 7.220209453803954e-05,
1203
- "loss": 0.016651667654514313,
1204
- "step": 830
1205
- },
1206
- {
1207
- "epoch": 0.5695770804911323,
1208
- "grad_norm": 0.8203716278076172,
1209
- "learning_rate": 7.216311002919064e-05,
1210
- "loss": 0.03519523441791535,
1211
- "step": 835
1212
- },
1213
- {
1214
- "epoch": 0.572987721691678,
1215
- "grad_norm": 1.957996129989624,
1216
- "learning_rate": 7.212386647501254e-05,
1217
- "loss": 0.03704521656036377,
1218
- "step": 840
1219
- },
1220
- {
1221
- "epoch": 0.5763983628922238,
1222
- "grad_norm": 0.2386065572500229,
1223
- "learning_rate": 7.208436416878125e-05,
1224
- "loss": 0.02845575213432312,
1225
- "step": 845
1226
- },
1227
- {
1228
- "epoch": 0.5798090040927695,
1229
- "grad_norm": 0.9101418256759644,
1230
- "learning_rate": 7.204460340570658e-05,
1231
- "loss": 0.01103741154074669,
1232
- "step": 850
1233
- },
1234
- {
1235
- "epoch": 0.5832196452933152,
1236
- "grad_norm": 0.22701004147529602,
1237
- "learning_rate": 7.200458448292972e-05,
1238
- "loss": 0.06184377670288086,
1239
- "step": 855
1240
- },
1241
- {
1242
- "epoch": 0.5866302864938608,
1243
- "grad_norm": 2.3091611862182617,
1244
- "learning_rate": 7.196430769952126e-05,
1245
- "loss": 0.0492431253194809,
1246
- "step": 860
1247
- },
1248
- {
1249
- "epoch": 0.5900409276944065,
1250
- "grad_norm": 0.6630973815917969,
1251
- "learning_rate": 7.192377335647876e-05,
1252
- "loss": 0.027876955270767213,
1253
- "step": 865
1254
- },
1255
- {
1256
- "epoch": 0.5934515688949522,
1257
- "grad_norm": 1.7400578260421753,
1258
- "learning_rate": 7.188298175672464e-05,
1259
- "loss": 0.023742210865020753,
1260
- "step": 870
1261
- },
1262
- {
1263
- "epoch": 0.596862210095498,
1264
- "grad_norm": 0.7121092081069946,
1265
- "learning_rate": 7.184193320510379e-05,
1266
- "loss": 0.02125793993473053,
1267
- "step": 875
1268
- },
1269
- {
1270
- "epoch": 0.6002728512960437,
1271
- "grad_norm": 0.429611474275589,
1272
- "learning_rate": 7.180062800838143e-05,
1273
- "loss": 0.06682519316673279,
1274
- "step": 880
1275
- },
1276
- {
1277
- "epoch": 0.6036834924965894,
1278
- "grad_norm": 0.018405891954898834,
1279
- "learning_rate": 7.17590664752407e-05,
1280
- "loss": 0.022519922256469725,
1281
- "step": 885
1282
- },
1283
- {
1284
- "epoch": 0.607094133697135,
1285
- "grad_norm": 0.9797202348709106,
1286
- "learning_rate": 7.171724891628046e-05,
1287
- "loss": 0.10513803958892823,
1288
- "step": 890
1289
- },
1290
- {
1291
- "epoch": 0.6105047748976807,
1292
- "grad_norm": 0.11931606382131577,
1293
- "learning_rate": 7.167517564401282e-05,
1294
- "loss": 0.055521953105926516,
1295
- "step": 895
1296
- },
1297
- {
1298
- "epoch": 0.6139154160982264,
1299
- "grad_norm": 0.04398813843727112,
1300
- "learning_rate": 7.163284697286097e-05,
1301
- "loss": 0.018342888355255126,
1302
- "step": 900
1303
- },
1304
- {
1305
- "epoch": 0.6173260572987722,
1306
- "grad_norm": 0.11505168676376343,
1307
- "learning_rate": 7.15902632191567e-05,
1308
- "loss": 0.026946401596069335,
1309
- "step": 905
1310
- },
1311
- {
1312
- "epoch": 0.6207366984993179,
1313
- "grad_norm": 0.3042922019958496,
1314
- "learning_rate": 7.154742470113816e-05,
1315
- "loss": 0.02314314842224121,
1316
- "step": 910
1317
- },
1318
- {
1319
- "epoch": 0.6241473396998636,
1320
- "grad_norm": 0.09922346472740173,
1321
- "learning_rate": 7.150433173894733e-05,
1322
- "loss": 0.06378543972969056,
1323
- "step": 915
1324
- },
1325
- {
1326
- "epoch": 0.6275579809004093,
1327
- "grad_norm": 0.6513009667396545,
1328
- "learning_rate": 7.146098465462776e-05,
1329
- "loss": 0.036993378400802614,
1330
- "step": 920
1331
- },
1332
- {
1333
- "epoch": 0.630968622100955,
1334
- "grad_norm": 1.131602168083191,
1335
- "learning_rate": 7.14173837721221e-05,
1336
- "loss": 0.09491010308265686,
1337
- "step": 925
1338
- },
1339
- {
1340
- "epoch": 0.6343792633015006,
1341
- "grad_norm": 0.1672857254743576,
1342
- "learning_rate": 7.137352941726969e-05,
1343
- "loss": 0.03719751834869385,
1344
- "step": 930
1345
- },
1346
- {
1347
- "epoch": 0.6377899045020464,
1348
- "grad_norm": 0.7450887560844421,
1349
- "learning_rate": 7.132942191780414e-05,
1350
- "loss": 0.028598809242248537,
1351
- "step": 935
1352
- },
1353
- {
1354
- "epoch": 0.6412005457025921,
1355
- "grad_norm": 2.3537657260894775,
1356
- "learning_rate": 7.128506160335084e-05,
1357
- "loss": 0.06678735613822936,
1358
- "step": 940
1359
- },
1360
- {
1361
- "epoch": 0.6446111869031378,
1362
- "grad_norm": 0.014428210444748402,
1363
- "learning_rate": 7.124044880542455e-05,
1364
- "loss": 0.018354634940624236,
1365
- "step": 945
1366
- },
1367
- {
1368
- "epoch": 0.6480218281036835,
1369
- "grad_norm": 2.9239041805267334,
1370
- "learning_rate": 7.119558385742688e-05,
1371
- "loss": 0.08242651224136352,
1372
- "step": 950
1373
- },
1374
- {
1375
- "epoch": 0.6514324693042292,
1376
- "grad_norm": 0.39032718539237976,
1377
- "learning_rate": 7.115046709464383e-05,
1378
- "loss": 0.023772728443145753,
1379
- "step": 955
1380
- },
1381
- {
1382
- "epoch": 0.654843110504775,
1383
- "grad_norm": 0.3000798523426056,
1384
- "learning_rate": 7.110509885424326e-05,
1385
- "loss": 0.03464276790618896,
1386
- "step": 960
1387
- },
1388
- {
1389
- "epoch": 0.6582537517053206,
1390
- "grad_norm": 0.3980049192905426,
1391
- "learning_rate": 7.105947947527238e-05,
1392
- "loss": 0.08540127277374268,
1393
- "step": 965
1394
- },
1395
- {
1396
- "epoch": 0.6616643929058663,
1397
- "grad_norm": 0.9492272734642029,
1398
- "learning_rate": 7.10136092986552e-05,
1399
- "loss": 0.02300785481929779,
1400
- "step": 970
1401
- },
1402
- {
1403
- "epoch": 0.665075034106412,
1404
- "grad_norm": 1.9585710763931274,
1405
- "learning_rate": 7.096748866719005e-05,
1406
- "loss": 0.034704044461250305,
1407
- "step": 975
1408
- },
1409
- {
1410
- "epoch": 0.6684856753069577,
1411
- "grad_norm": 1.142238974571228,
1412
- "learning_rate": 7.092111792554689e-05,
1413
- "loss": 0.01860647052526474,
1414
- "step": 980
1415
- },
1416
- {
1417
- "epoch": 0.6718963165075034,
1418
- "grad_norm": 0.8443534970283508,
1419
- "learning_rate": 7.087449742026488e-05,
1420
- "loss": 0.03302992284297943,
1421
- "step": 985
1422
- },
1423
- {
1424
- "epoch": 0.6753069577080492,
1425
- "grad_norm": 1.0402863025665283,
1426
- "learning_rate": 7.082762749974968e-05,
1427
- "loss": 0.03963000178337097,
1428
- "step": 990
1429
- },
1430
- {
1431
- "epoch": 0.6787175989085948,
1432
- "grad_norm": 0.11959892511367798,
1433
- "learning_rate": 7.078050851427089e-05,
1434
- "loss": 0.0187692254781723,
1435
- "step": 995
1436
- },
1437
- {
1438
- "epoch": 0.6821282401091405,
1439
- "grad_norm": 1.495011329650879,
1440
- "learning_rate": 7.073314081595945e-05,
1441
- "loss": 0.050608736276626584,
1442
- "step": 1000
1443
- },
1444
- {
1445
- "epoch": 0.6855388813096862,
1446
- "grad_norm": 0.2534504234790802,
1447
- "learning_rate": 7.068552475880499e-05,
1448
- "loss": 0.0076520174741745,
1449
- "step": 1005
1450
- },
1451
- {
1452
- "epoch": 0.6889495225102319,
1453
- "grad_norm": 0.17387191951274872,
1454
- "learning_rate": 7.063766069865314e-05,
1455
- "loss": 0.028283193707466125,
1456
- "step": 1010
1457
- },
1458
- {
1459
- "epoch": 0.6923601637107776,
1460
- "grad_norm": 0.07424458116292953,
1461
- "learning_rate": 7.058954899320297e-05,
1462
- "loss": 0.03078552782535553,
1463
- "step": 1015
1464
- },
1465
- {
1466
- "epoch": 0.6957708049113234,
1467
- "grad_norm": 0.5854848027229309,
1468
- "learning_rate": 7.05411900020042e-05,
1469
- "loss": 0.02033105492591858,
1470
- "step": 1020
1471
- },
1472
- {
1473
- "epoch": 0.699181446111869,
1474
- "grad_norm": 0.09108871966600418,
1475
- "learning_rate": 7.049258408645463e-05,
1476
- "loss": 0.0510578989982605,
1477
- "step": 1025
1478
- },
1479
- {
1480
- "epoch": 0.7025920873124147,
1481
- "grad_norm": 0.2851751148700714,
1482
- "learning_rate": 7.044373160979734e-05,
1483
- "loss": 0.10413439273834228,
1484
- "step": 1030
1485
- },
1486
- {
1487
- "epoch": 0.7060027285129604,
1488
- "grad_norm": 0.8032590746879578,
1489
- "learning_rate": 7.039463293711804e-05,
1490
- "loss": 0.05853385329246521,
1491
- "step": 1035
1492
- },
1493
- {
1494
- "epoch": 0.7094133697135061,
1495
- "grad_norm": 0.1301775723695755,
1496
- "learning_rate": 7.03452884353423e-05,
1497
- "loss": 0.051472657918930055,
1498
- "step": 1040
1499
- },
1500
- {
1501
- "epoch": 0.7128240109140518,
1502
- "grad_norm": 0.46156957745552063,
1503
- "learning_rate": 7.029569847323287e-05,
1504
- "loss": 0.034115567803382874,
1505
- "step": 1045
1506
- },
1507
- {
1508
- "epoch": 0.7162346521145976,
1509
- "grad_norm": 1.081560730934143,
1510
- "learning_rate": 7.02458634213868e-05,
1511
- "loss": 0.059652507305145264,
1512
- "step": 1050
1513
- },
1514
- {
1515
- "epoch": 0.7196452933151433,
1516
- "grad_norm": 0.721208930015564,
1517
- "learning_rate": 7.019578365223286e-05,
1518
- "loss": 0.061070340871810916,
1519
- "step": 1055
1520
- },
1521
- {
1522
- "epoch": 0.723055934515689,
1523
- "grad_norm": 0.5738947987556458,
1524
- "learning_rate": 7.014545954002855e-05,
1525
- "loss": 0.03556577265262604,
1526
- "step": 1060
1527
- },
1528
- {
1529
- "epoch": 0.7264665757162346,
1530
- "grad_norm": 0.15053460001945496,
1531
- "learning_rate": 7.009489146085744e-05,
1532
- "loss": 0.03284372091293335,
1533
- "step": 1065
1534
- },
1535
- {
1536
- "epoch": 0.7298772169167803,
1537
- "grad_norm": 0.4496553838253021,
1538
- "learning_rate": 7.004407979262635e-05,
1539
- "loss": 0.07945018410682678,
1540
- "step": 1070
1541
- },
1542
- {
1543
- "epoch": 0.7332878581173261,
1544
- "grad_norm": 1.1821213960647583,
1545
- "learning_rate": 6.999302491506245e-05,
1546
- "loss": 0.033741748332977294,
1547
- "step": 1075
1548
- },
1549
- {
1550
- "epoch": 0.7366984993178718,
1551
- "grad_norm": 0.2809429168701172,
1552
- "learning_rate": 6.994172720971047e-05,
1553
- "loss": 0.023005199432373048,
1554
- "step": 1080
1555
- },
1556
- {
1557
- "epoch": 0.7401091405184175,
1558
- "grad_norm": 0.14925819635391235,
1559
- "learning_rate": 6.989018705992991e-05,
1560
- "loss": 0.01791207939386368,
1561
- "step": 1085
1562
- },
1563
- {
1564
- "epoch": 0.7435197817189632,
1565
- "grad_norm": 1.1947131156921387,
1566
- "learning_rate": 6.983840485089203e-05,
1567
- "loss": 0.03395574688911438,
1568
- "step": 1090
1569
- },
1570
- {
1571
- "epoch": 0.7469304229195088,
1572
- "grad_norm": 1.336547613143921,
1573
- "learning_rate": 6.978638096957712e-05,
1574
- "loss": 0.02712726593017578,
1575
- "step": 1095
1576
- },
1577
- {
1578
- "epoch": 0.7489768076398363,
1579
- "eval_loss": 0.05635881423950195,
1580
- "eval_runtime": 0.8951,
1581
- "eval_samples_per_second": 83.789,
1582
- "eval_steps_per_second": 2.234,
1583
- "step": 1098
1584
- },
1585
- {
1586
- "eval_cer_subset": 0.023981468864967978,
1587
- "eval_cer_subset_edit_distance": 176,
1588
- "eval_cer_subset_groups": 75,
1589
- "eval_cer_subset_items": 75,
1590
- "eval_cer_subset_ref_chars": 7339,
1591
- "step": 1098
1592
- }
1593
- ],
1594
- "logging_steps": 5,
1595
- "max_steps": 5864,
1596
- "num_input_tokens_seen": 0,
1597
- "num_train_epochs": 4,
1598
- "save_steps": 366,
1599
- "stateful_callbacks": {
1600
- "TrainerControl": {
1601
- "args": {
1602
- "should_epoch_stop": false,
1603
- "should_evaluate": false,
1604
- "should_log": false,
1605
- "should_save": true,
1606
- "should_training_stop": false
1607
- },
1608
- "attributes": {}
1609
- }
1610
- },
1611
- "total_flos": 1.1443907853139968e+16,
1612
- "train_batch_size": 2,
1613
- "trial_name": null,
1614
- "trial_params": null
1615
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1098/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bc6915d8d9dd9b5c9c17756c87ba7ec8221fd06789232d79225f9518167f0aa1
3
- size 5841
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/README.md DELETED
@@ -1,210 +0,0 @@
1
- ---
2
- base_model: unsloth/Meta-Llama-3.1-8B
3
- library_name: peft
4
- pipeline_tag: text-generation
5
- tags:
6
- - base_model:adapter:unsloth/Meta-Llama-3.1-8B
7
- - lora
8
- - sft
9
- - transformers
10
- - trl
11
- - unsloth
12
- ---
13
-
14
- # Model Card for Model ID
15
-
16
- <!-- Provide a quick summary of what the model is/does. -->
17
-
18
-
19
-
20
- ## Model Details
21
-
22
- ### Model Description
23
-
24
- <!-- Provide a longer summary of what this model is. -->
25
-
26
-
27
-
28
- - **Developed by:** [More Information Needed]
29
- - **Funded by [optional]:** [More Information Needed]
30
- - **Shared by [optional]:** [More Information Needed]
31
- - **Model type:** [More Information Needed]
32
- - **Language(s) (NLP):** [More Information Needed]
33
- - **License:** [More Information Needed]
34
- - **Finetuned from model [optional]:** [More Information Needed]
35
-
36
- ### Model Sources [optional]
37
-
38
- <!-- Provide the basic links for the model. -->
39
-
40
- - **Repository:** [More Information Needed]
41
- - **Paper [optional]:** [More Information Needed]
42
- - **Demo [optional]:** [More Information Needed]
43
-
44
- ## Uses
45
-
46
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
-
48
- ### Direct Use
49
-
50
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
-
52
- [More Information Needed]
53
-
54
- ### Downstream Use [optional]
55
-
56
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
-
58
- [More Information Needed]
59
-
60
- ### Out-of-Scope Use
61
-
62
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
-
64
- [More Information Needed]
65
-
66
- ## Bias, Risks, and Limitations
67
-
68
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
-
70
- [More Information Needed]
71
-
72
- ### Recommendations
73
-
74
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
-
76
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
-
78
- ## How to Get Started with the Model
79
-
80
- Use the code below to get started with the model.
81
-
82
- [More Information Needed]
83
-
84
- ## Training Details
85
-
86
- ### Training Data
87
-
88
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
-
90
- [More Information Needed]
91
-
92
- ### Training Procedure
93
-
94
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
-
96
- #### Preprocessing [optional]
97
-
98
- [More Information Needed]
99
-
100
-
101
- #### Training Hyperparameters
102
-
103
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
-
105
- #### Speeds, Sizes, Times [optional]
106
-
107
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
-
109
- [More Information Needed]
110
-
111
- ## Evaluation
112
-
113
- <!-- This section describes the evaluation protocols and provides the results. -->
114
-
115
- ### Testing Data, Factors & Metrics
116
-
117
- #### Testing Data
118
-
119
- <!-- This should link to a Dataset Card if possible. -->
120
-
121
- [More Information Needed]
122
-
123
- #### Factors
124
-
125
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
-
127
- [More Information Needed]
128
-
129
- #### Metrics
130
-
131
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
-
133
- [More Information Needed]
134
-
135
- ### Results
136
-
137
- [More Information Needed]
138
-
139
- #### Summary
140
-
141
-
142
-
143
- ## Model Examination [optional]
144
-
145
- <!-- Relevant interpretability work for the model goes here -->
146
-
147
- [More Information Needed]
148
-
149
- ## Environmental Impact
150
-
151
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
-
153
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
-
155
- - **Hardware Type:** [More Information Needed]
156
- - **Hours used:** [More Information Needed]
157
- - **Cloud Provider:** [More Information Needed]
158
- - **Compute Region:** [More Information Needed]
159
- - **Carbon Emitted:** [More Information Needed]
160
-
161
- ## Technical Specifications [optional]
162
-
163
- ### Model Architecture and Objective
164
-
165
- [More Information Needed]
166
-
167
- ### Compute Infrastructure
168
-
169
- [More Information Needed]
170
-
171
- #### Hardware
172
-
173
- [More Information Needed]
174
-
175
- #### Software
176
-
177
- [More Information Needed]
178
-
179
- ## Citation [optional]
180
-
181
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
-
183
- **BibTeX:**
184
-
185
- [More Information Needed]
186
-
187
- **APA:**
188
-
189
- [More Information Needed]
190
-
191
- ## Glossary [optional]
192
-
193
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
-
195
- [More Information Needed]
196
-
197
- ## More Information [optional]
198
-
199
- [More Information Needed]
200
-
201
- ## Model Card Authors [optional]
202
-
203
- [More Information Needed]
204
-
205
- ## Model Card Contact
206
-
207
- [More Information Needed]
208
- ### Framework versions
209
-
210
- - PEFT 0.18.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/adapter_config.json DELETED
@@ -1,50 +0,0 @@
1
- {
2
- "alora_invocation_tokens": null,
3
- "alpha_pattern": {},
4
- "arrow_config": null,
5
- "auto_mapping": {
6
- "base_model_class": "LlamaForCausalLM",
7
- "parent_library": "transformers.models.llama.modeling_llama",
8
- "unsloth_fixed": true
9
- },
10
- "base_model_name_or_path": "unsloth/Meta-Llama-3.1-8B",
11
- "bias": "none",
12
- "corda_config": null,
13
- "ensure_weight_tying": false,
14
- "eva_config": null,
15
- "exclude_modules": null,
16
- "fan_in_fan_out": false,
17
- "inference_mode": true,
18
- "init_lora_weights": true,
19
- "layer_replication": null,
20
- "layers_pattern": null,
21
- "layers_to_transform": null,
22
- "loftq_config": {},
23
- "lora_alpha": 16,
24
- "lora_bias": false,
25
- "lora_dropout": 0.0,
26
- "megatron_config": null,
27
- "megatron_core": "megatron.core",
28
- "modules_to_save": null,
29
- "peft_type": "LORA",
30
- "peft_version": "0.18.1",
31
- "qalora_group_size": 16,
32
- "r": 8,
33
- "rank_pattern": {},
34
- "revision": null,
35
- "target_modules": [
36
- "down_proj",
37
- "v_proj",
38
- "gate_proj",
39
- "up_proj",
40
- "o_proj",
41
- "q_proj",
42
- "k_proj"
43
- ],
44
- "target_parameters": null,
45
- "task_type": "CAUSAL_LM",
46
- "trainable_token_indices": null,
47
- "use_dora": false,
48
- "use_qalora": false,
49
- "use_rslora": false
50
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c7d76ec612c68701e4ab38fd7332e83b533d3cb5d77e47f5f177c8f5e3f02371
3
- size 83945296
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/chat_template.jinja DELETED
@@ -1,12 +0,0 @@
1
- {{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ messages[0]['content'] + '
2
-
3
- ' }}{% set loop_messages = messages[1:] %}{% else %}{{ '' + '
4
-
5
- ' }}{% set loop_messages = messages %}{% endif %}{% for message in loop_messages %}{% if message['role'] == 'user' %}{{ '### Instruction:
6
- ' + message['content'] + '
7
-
8
- ' }}{% elif message['role'] == 'assistant' %}{{ '### Response:
9
- ' + message['content'] + eos_token + '
10
-
11
- ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '### Response:
12
- ' }}{% endif %}
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:573920222e0747f8b4f52f63492bebb39a8010748f870f74d36be47092625986
3
- size 43127973
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:55618d72fdc98ce531c1d13612b817a14a50cd3ee258c44dec6a79dee2ab54ab
3
- size 14645
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:092b2ba9e97dcb53fb28563a3b62dc47503e6a9c063fdb13a4250a47b2fa6abf
3
- size 1465
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/tokenizer.json DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
3
- size 17209920
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/tokenizer_config.json DELETED
@@ -1,18 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "bos_token": "<|begin_of_text|>",
4
- "clean_up_tokenization_spaces": true,
5
- "eos_token": "<|end_of_text|>",
6
- "from_slow": true,
7
- "is_local": false,
8
- "legacy": false,
9
- "model_input_names": [
10
- "input_ids",
11
- "attention_mask"
12
- ],
13
- "model_max_length": 131072,
14
- "pad_token": "<|finetune_right_pad_id|>",
15
- "padding_side": "right",
16
- "tokenizer_class": "TokenizersBackend",
17
- "unk_token": null
18
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/trainer_state.json DELETED
The diff for this file is too large to render. See raw diff
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-11728/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6adec376d54028ef57ef3dc856a5cba12bab9c0d580369637fa983b6072064f7
3
- size 5841
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/README.md DELETED
@@ -1,210 +0,0 @@
1
- ---
2
- base_model: unsloth/Meta-Llama-3.1-8B
3
- library_name: peft
4
- pipeline_tag: text-generation
5
- tags:
6
- - base_model:adapter:unsloth/Meta-Llama-3.1-8B
7
- - lora
8
- - sft
9
- - transformers
10
- - trl
11
- - unsloth
12
- ---
13
-
14
- # Model Card for Model ID
15
-
16
- <!-- Provide a quick summary of what the model is/does. -->
17
-
18
-
19
-
20
- ## Model Details
21
-
22
- ### Model Description
23
-
24
- <!-- Provide a longer summary of what this model is. -->
25
-
26
-
27
-
28
- - **Developed by:** [More Information Needed]
29
- - **Funded by [optional]:** [More Information Needed]
30
- - **Shared by [optional]:** [More Information Needed]
31
- - **Model type:** [More Information Needed]
32
- - **Language(s) (NLP):** [More Information Needed]
33
- - **License:** [More Information Needed]
34
- - **Finetuned from model [optional]:** [More Information Needed]
35
-
36
- ### Model Sources [optional]
37
-
38
- <!-- Provide the basic links for the model. -->
39
-
40
- - **Repository:** [More Information Needed]
41
- - **Paper [optional]:** [More Information Needed]
42
- - **Demo [optional]:** [More Information Needed]
43
-
44
- ## Uses
45
-
46
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
-
48
- ### Direct Use
49
-
50
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
-
52
- [More Information Needed]
53
-
54
- ### Downstream Use [optional]
55
-
56
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
-
58
- [More Information Needed]
59
-
60
- ### Out-of-Scope Use
61
-
62
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
-
64
- [More Information Needed]
65
-
66
- ## Bias, Risks, and Limitations
67
-
68
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
-
70
- [More Information Needed]
71
-
72
- ### Recommendations
73
-
74
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
-
76
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
-
78
- ## How to Get Started with the Model
79
-
80
- Use the code below to get started with the model.
81
-
82
- [More Information Needed]
83
-
84
- ## Training Details
85
-
86
- ### Training Data
87
-
88
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
-
90
- [More Information Needed]
91
-
92
- ### Training Procedure
93
-
94
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
-
96
- #### Preprocessing [optional]
97
-
98
- [More Information Needed]
99
-
100
-
101
- #### Training Hyperparameters
102
-
103
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
-
105
- #### Speeds, Sizes, Times [optional]
106
-
107
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
-
109
- [More Information Needed]
110
-
111
- ## Evaluation
112
-
113
- <!-- This section describes the evaluation protocols and provides the results. -->
114
-
115
- ### Testing Data, Factors & Metrics
116
-
117
- #### Testing Data
118
-
119
- <!-- This should link to a Dataset Card if possible. -->
120
-
121
- [More Information Needed]
122
-
123
- #### Factors
124
-
125
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
-
127
- [More Information Needed]
128
-
129
- #### Metrics
130
-
131
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
-
133
- [More Information Needed]
134
-
135
- ### Results
136
-
137
- [More Information Needed]
138
-
139
- #### Summary
140
-
141
-
142
-
143
- ## Model Examination [optional]
144
-
145
- <!-- Relevant interpretability work for the model goes here -->
146
-
147
- [More Information Needed]
148
-
149
- ## Environmental Impact
150
-
151
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
-
153
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
-
155
- - **Hardware Type:** [More Information Needed]
156
- - **Hours used:** [More Information Needed]
157
- - **Cloud Provider:** [More Information Needed]
158
- - **Compute Region:** [More Information Needed]
159
- - **Carbon Emitted:** [More Information Needed]
160
-
161
- ## Technical Specifications [optional]
162
-
163
- ### Model Architecture and Objective
164
-
165
- [More Information Needed]
166
-
167
- ### Compute Infrastructure
168
-
169
- [More Information Needed]
170
-
171
- #### Hardware
172
-
173
- [More Information Needed]
174
-
175
- #### Software
176
-
177
- [More Information Needed]
178
-
179
- ## Citation [optional]
180
-
181
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
-
183
- **BibTeX:**
184
-
185
- [More Information Needed]
186
-
187
- **APA:**
188
-
189
- [More Information Needed]
190
-
191
- ## Glossary [optional]
192
-
193
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
-
195
- [More Information Needed]
196
-
197
- ## More Information [optional]
198
-
199
- [More Information Needed]
200
-
201
- ## Model Card Authors [optional]
202
-
203
- [More Information Needed]
204
-
205
- ## Model Card Contact
206
-
207
- [More Information Needed]
208
- ### Framework versions
209
-
210
- - PEFT 0.18.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/adapter_config.json DELETED
@@ -1,50 +0,0 @@
1
- {
2
- "alora_invocation_tokens": null,
3
- "alpha_pattern": {},
4
- "arrow_config": null,
5
- "auto_mapping": {
6
- "base_model_class": "LlamaForCausalLM",
7
- "parent_library": "transformers.models.llama.modeling_llama",
8
- "unsloth_fixed": true
9
- },
10
- "base_model_name_or_path": "unsloth/Meta-Llama-3.1-8B",
11
- "bias": "none",
12
- "corda_config": null,
13
- "ensure_weight_tying": false,
14
- "eva_config": null,
15
- "exclude_modules": null,
16
- "fan_in_fan_out": false,
17
- "inference_mode": true,
18
- "init_lora_weights": true,
19
- "layer_replication": null,
20
- "layers_pattern": null,
21
- "layers_to_transform": null,
22
- "loftq_config": {},
23
- "lora_alpha": 16,
24
- "lora_bias": false,
25
- "lora_dropout": 0.0,
26
- "megatron_config": null,
27
- "megatron_core": "megatron.core",
28
- "modules_to_save": null,
29
- "peft_type": "LORA",
30
- "peft_version": "0.18.1",
31
- "qalora_group_size": 16,
32
- "r": 8,
33
- "rank_pattern": {},
34
- "revision": null,
35
- "target_modules": [
36
- "q_proj",
37
- "k_proj",
38
- "up_proj",
39
- "gate_proj",
40
- "down_proj",
41
- "o_proj",
42
- "v_proj"
43
- ],
44
- "target_parameters": null,
45
- "task_type": "CAUSAL_LM",
46
- "trainable_token_indices": null,
47
- "use_dora": false,
48
- "use_qalora": false,
49
- "use_rslora": false
50
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:251f115f1399311d664d30a65f3f69edd39ca1066ff9de11cf1af58abc53789c
3
- size 83945296
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/chat_template.jinja DELETED
@@ -1,12 +0,0 @@
1
- {{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ messages[0]['content'] + '
2
-
3
- ' }}{% set loop_messages = messages[1:] %}{% else %}{{ '' + '
4
-
5
- ' }}{% set loop_messages = messages %}{% endif %}{% for message in loop_messages %}{% if message['role'] == 'user' %}{{ '### Instruction:
6
- ' + message['content'] + '
7
-
8
- ' }}{% elif message['role'] == 'assistant' %}{{ '### Response:
9
- ' + message['content'] + eos_token + '
10
-
11
- ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '### Response:
12
- ' }}{% endif %}
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:5571e8cb2c75967acf8b8fe019749ea08c8733a3dc2fd2a8e0f667a32a4c9f57
3
- size 43127973
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:4edcb0c400e489258895e3e8e12e12a56c067fbaee316ef0aecc58f56df05ed8
3
- size 14645
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:0b7686410b47d9ea46c29370951c2c9ee4eb0cc9deb04bed685304ede7411208
3
- size 1465
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/tokenizer.json DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
3
- size 17209920
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/tokenizer_config.json DELETED
@@ -1,18 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "bos_token": "<|begin_of_text|>",
4
- "clean_up_tokenization_spaces": true,
5
- "eos_token": "<|end_of_text|>",
6
- "from_slow": true,
7
- "is_local": false,
8
- "legacy": false,
9
- "model_input_names": [
10
- "input_ids",
11
- "attention_mask"
12
- ],
13
- "model_max_length": 131072,
14
- "pad_token": "<|finetune_right_pad_id|>",
15
- "padding_side": "right",
16
- "tokenizer_class": "TokenizersBackend",
17
- "unk_token": null
18
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/trainer_state.json DELETED
@@ -1,2142 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 0.9986357435197817,
6
- "eval_steps": 366,
7
- "global_step": 1464,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.0034106412005457027,
14
- "grad_norm": 3.1571648120880127,
15
- "learning_rate": 2.542372881355932e-06,
16
- "loss": 1.6981744766235352,
17
- "step": 5
18
- },
19
- {
20
- "epoch": 0.0068212824010914054,
21
- "grad_norm": 9.80073070526123,
22
- "learning_rate": 5.720338983050847e-06,
23
- "loss": 1.5801130294799806,
24
- "step": 10
25
- },
26
- {
27
- "epoch": 0.010231923601637109,
28
- "grad_norm": 4.481558799743652,
29
- "learning_rate": 8.898305084745763e-06,
30
- "loss": 2.1718717575073243,
31
- "step": 15
32
- },
33
- {
34
- "epoch": 0.013642564802182811,
35
- "grad_norm": 7.221553802490234,
36
- "learning_rate": 1.2076271186440677e-05,
37
- "loss": 1.5270480155944823,
38
- "step": 20
39
- },
40
- {
41
- "epoch": 0.017053206002728513,
42
- "grad_norm": 5.330269813537598,
43
- "learning_rate": 1.5254237288135592e-05,
44
- "loss": 0.9586630821228027,
45
- "step": 25
46
- },
47
- {
48
- "epoch": 0.020463847203274217,
49
- "grad_norm": 1.7404323816299438,
50
- "learning_rate": 1.8432203389830506e-05,
51
- "loss": 0.48505802154541017,
52
- "step": 30
53
- },
54
- {
55
- "epoch": 0.023874488403819918,
56
- "grad_norm": 1.2418887615203857,
57
- "learning_rate": 2.1610169491525424e-05,
58
- "loss": 0.24452033042907714,
59
- "step": 35
60
- },
61
- {
62
- "epoch": 0.027285129604365622,
63
- "grad_norm": 1.5928819179534912,
64
- "learning_rate": 2.4788135593220338e-05,
65
- "loss": 0.25337533950805663,
66
- "step": 40
67
- },
68
- {
69
- "epoch": 0.030695770804911322,
70
- "grad_norm": 2.0335769653320312,
71
- "learning_rate": 2.796610169491525e-05,
72
- "loss": 0.1938886046409607,
73
- "step": 45
74
- },
75
- {
76
- "epoch": 0.034106412005457026,
77
- "grad_norm": 0.18029411137104034,
78
- "learning_rate": 3.114406779661017e-05,
79
- "loss": 0.1834133505821228,
80
- "step": 50
81
- },
82
- {
83
- "epoch": 0.03751705320600273,
84
- "grad_norm": 1.7757192850112915,
85
- "learning_rate": 3.432203389830508e-05,
86
- "loss": 0.15151114463806153,
87
- "step": 55
88
- },
89
- {
90
- "epoch": 0.040927694406548434,
91
- "grad_norm": 2.1376893520355225,
92
- "learning_rate": 3.75e-05,
93
- "loss": 0.21634674072265625,
94
- "step": 60
95
- },
96
- {
97
- "epoch": 0.04433833560709413,
98
- "grad_norm": 2.196387767791748,
99
- "learning_rate": 4.067796610169491e-05,
100
- "loss": 0.11320395469665527,
101
- "step": 65
102
- },
103
- {
104
- "epoch": 0.047748976807639835,
105
- "grad_norm": 1.5888807773590088,
106
- "learning_rate": 4.3855932203389825e-05,
107
- "loss": 0.11050152778625488,
108
- "step": 70
109
- },
110
- {
111
- "epoch": 0.05115961800818554,
112
- "grad_norm": 4.606944561004639,
113
- "learning_rate": 4.703389830508474e-05,
114
- "loss": 0.23255281448364257,
115
- "step": 75
116
- },
117
- {
118
- "epoch": 0.054570259208731244,
119
- "grad_norm": 1.7308639287948608,
120
- "learning_rate": 5.021186440677966e-05,
121
- "loss": 0.08995866179466247,
122
- "step": 80
123
- },
124
- {
125
- "epoch": 0.05798090040927694,
126
- "grad_norm": 2.622438430786133,
127
- "learning_rate": 5.338983050847457e-05,
128
- "loss": 0.05715223550796509,
129
- "step": 85
130
- },
131
- {
132
- "epoch": 0.061391541609822645,
133
- "grad_norm": 0.34140461683273315,
134
- "learning_rate": 5.656779661016949e-05,
135
- "loss": 0.06703727245330811,
136
- "step": 90
137
- },
138
- {
139
- "epoch": 0.06480218281036836,
140
- "grad_norm": 2.7264959812164307,
141
- "learning_rate": 5.97457627118644e-05,
142
- "loss": 0.10092716217041016,
143
- "step": 95
144
- },
145
- {
146
- "epoch": 0.06821282401091405,
147
- "grad_norm": 2.0234780311584473,
148
- "learning_rate": 6.292372881355932e-05,
149
- "loss": 0.03579937815666199,
150
- "step": 100
151
- },
152
- {
153
- "epoch": 0.07162346521145975,
154
- "grad_norm": 1.3716133832931519,
155
- "learning_rate": 6.610169491525423e-05,
156
- "loss": 0.07851418852806091,
157
- "step": 105
158
- },
159
- {
160
- "epoch": 0.07503410641200546,
161
- "grad_norm": 1.2307227849960327,
162
- "learning_rate": 6.927966101694914e-05,
163
- "loss": 0.07370938658714295,
164
- "step": 110
165
- },
166
- {
167
- "epoch": 0.07844474761255116,
168
- "grad_norm": 1.7142690420150757,
169
- "learning_rate": 7.245762711864406e-05,
170
- "loss": 0.08064572811126709,
171
- "step": 115
172
- },
173
- {
174
- "epoch": 0.08185538881309687,
175
- "grad_norm": 1.343595266342163,
176
- "learning_rate": 7.499999439507554e-05,
177
- "loss": 0.11841402053833008,
178
- "step": 120
179
- },
180
- {
181
- "epoch": 0.08526603001364257,
182
- "grad_norm": 1.2511327266693115,
183
- "learning_rate": 7.499979822289558e-05,
184
- "loss": 0.06974860429763793,
185
- "step": 125
186
- },
187
- {
188
- "epoch": 0.08867667121418826,
189
- "grad_norm": 1.5642656087875366,
190
- "learning_rate": 7.49993218061684e-05,
191
- "loss": 0.10972714424133301,
192
- "step": 130
193
- },
194
- {
195
- "epoch": 0.09208731241473397,
196
- "grad_norm": 1.6669789552688599,
197
- "learning_rate": 7.499856514845436e-05,
198
- "loss": 0.09864612817764282,
199
- "step": 135
200
- },
201
- {
202
- "epoch": 0.09549795361527967,
203
- "grad_norm": 0.9789333343505859,
204
- "learning_rate": 7.499752825540815e-05,
205
- "loss": 0.0699621558189392,
206
- "step": 140
207
- },
208
- {
209
- "epoch": 0.09890859481582538,
210
- "grad_norm": 0.41792476177215576,
211
- "learning_rate": 7.499621113477873e-05,
212
- "loss": 0.06734349727630615,
213
- "step": 145
214
- },
215
- {
216
- "epoch": 0.10231923601637108,
217
- "grad_norm": 1.5968533754348755,
218
- "learning_rate": 7.499461379640919e-05,
219
- "loss": 0.060729533433914185,
220
- "step": 150
221
- },
222
- {
223
- "epoch": 0.10572987721691678,
224
- "grad_norm": 0.8512193560600281,
225
- "learning_rate": 7.499273625223683e-05,
226
- "loss": 0.052730172872543335,
227
- "step": 155
228
- },
229
- {
230
- "epoch": 0.10914051841746249,
231
- "grad_norm": 1.3257211446762085,
232
- "learning_rate": 7.499057851629299e-05,
233
- "loss": 0.10094538927078248,
234
- "step": 160
235
- },
236
- {
237
- "epoch": 0.11255115961800818,
238
- "grad_norm": 0.659118115901947,
239
- "learning_rate": 7.498814060470288e-05,
240
- "loss": 0.01604635864496231,
241
- "step": 165
242
- },
243
- {
244
- "epoch": 0.11596180081855388,
245
- "grad_norm": 2.454979181289673,
246
- "learning_rate": 7.49854225356856e-05,
247
- "loss": 0.13272385597229003,
248
- "step": 170
249
- },
250
- {
251
- "epoch": 0.11937244201909959,
252
- "grad_norm": 2.510040521621704,
253
- "learning_rate": 7.498242432955388e-05,
254
- "loss": 0.08396227955818177,
255
- "step": 175
256
- },
257
- {
258
- "epoch": 0.12278308321964529,
259
- "grad_norm": 0.2637259364128113,
260
- "learning_rate": 7.4979146008714e-05,
261
- "loss": 0.05852065086364746,
262
- "step": 180
263
- },
264
- {
265
- "epoch": 0.126193724420191,
266
- "grad_norm": 1.4320851564407349,
267
- "learning_rate": 7.497558759766564e-05,
268
- "loss": 0.10392802953720093,
269
- "step": 185
270
- },
271
- {
272
- "epoch": 0.1296043656207367,
273
- "grad_norm": 1.273027777671814,
274
- "learning_rate": 7.497174912300156e-05,
275
- "loss": 0.08062989711761474,
276
- "step": 190
277
- },
278
- {
279
- "epoch": 0.1330150068212824,
280
- "grad_norm": 0.8102871179580688,
281
- "learning_rate": 7.496763061340759e-05,
282
- "loss": 0.07294153571128845,
283
- "step": 195
284
- },
285
- {
286
- "epoch": 0.1364256480218281,
287
- "grad_norm": 1.951328992843628,
288
- "learning_rate": 7.496323209966228e-05,
289
- "loss": 0.07738351821899414,
290
- "step": 200
291
- },
292
- {
293
- "epoch": 0.13983628922237382,
294
- "grad_norm": 0.3880983889102936,
295
- "learning_rate": 7.495855361463674e-05,
296
- "loss": 0.07225048542022705,
297
- "step": 205
298
- },
299
- {
300
- "epoch": 0.1432469304229195,
301
- "grad_norm": 3.3205058574676514,
302
- "learning_rate": 7.495359519329433e-05,
303
- "loss": 0.05682974457740784,
304
- "step": 210
305
- },
306
- {
307
- "epoch": 0.1466575716234652,
308
- "grad_norm": 0.9203559160232544,
309
- "learning_rate": 7.49483568726905e-05,
310
- "loss": 0.08767472505569458,
311
- "step": 215
312
- },
313
- {
314
- "epoch": 0.15006821282401092,
315
- "grad_norm": 0.585360586643219,
316
- "learning_rate": 7.494283869197239e-05,
317
- "loss": 0.039227068424224854,
318
- "step": 220
319
- },
320
- {
321
- "epoch": 0.1534788540245566,
322
- "grad_norm": 1.7096059322357178,
323
- "learning_rate": 7.493704069237862e-05,
324
- "loss": 0.10281096696853638,
325
- "step": 225
326
- },
327
- {
328
- "epoch": 0.15688949522510232,
329
- "grad_norm": 0.4110204875469208,
330
- "learning_rate": 7.493096291723898e-05,
331
- "loss": 0.04346161186695099,
332
- "step": 230
333
- },
334
- {
335
- "epoch": 0.16030013642564803,
336
- "grad_norm": 1.3272292613983154,
337
- "learning_rate": 7.492460541197404e-05,
338
- "loss": 0.049719154834747314,
339
- "step": 235
340
- },
341
- {
342
- "epoch": 0.16371077762619374,
343
- "grad_norm": 1.1005016565322876,
344
- "learning_rate": 7.491796822409494e-05,
345
- "loss": 0.09335108399391175,
346
- "step": 240
347
- },
348
- {
349
- "epoch": 0.16712141882673942,
350
- "grad_norm": 0.7811501026153564,
351
- "learning_rate": 7.491105140320285e-05,
352
- "loss": 0.05943926572799683,
353
- "step": 245
354
- },
355
- {
356
- "epoch": 0.17053206002728513,
357
- "grad_norm": 1.4607417583465576,
358
- "learning_rate": 7.490385500098879e-05,
359
- "loss": 0.04385361075401306,
360
- "step": 250
361
- },
362
- {
363
- "epoch": 0.17394270122783084,
364
- "grad_norm": 0.394960880279541,
365
- "learning_rate": 7.489637907123308e-05,
366
- "loss": 0.04446137547492981,
367
- "step": 255
368
- },
369
- {
370
- "epoch": 0.17735334242837653,
371
- "grad_norm": 0.8768635988235474,
372
- "learning_rate": 7.488862366980505e-05,
373
- "loss": 0.04143576025962829,
374
- "step": 260
375
- },
376
- {
377
- "epoch": 0.18076398362892224,
378
- "grad_norm": 1.9996010065078735,
379
- "learning_rate": 7.488058885466262e-05,
380
- "loss": 0.07952215671539306,
381
- "step": 265
382
- },
383
- {
384
- "epoch": 0.18417462482946795,
385
- "grad_norm": 0.03770223259925842,
386
- "learning_rate": 7.487227468585178e-05,
387
- "loss": 0.02531362771987915,
388
- "step": 270
389
- },
390
- {
391
- "epoch": 0.18758526603001363,
392
- "grad_norm": 0.26082542538642883,
393
- "learning_rate": 7.486368122550619e-05,
394
- "loss": 0.09930967688560485,
395
- "step": 275
396
- },
397
- {
398
- "epoch": 0.19099590723055934,
399
- "grad_norm": 5.622270584106445,
400
- "learning_rate": 7.485480853784677e-05,
401
- "loss": 0.06534865498542786,
402
- "step": 280
403
- },
404
- {
405
- "epoch": 0.19440654843110505,
406
- "grad_norm": 0.5298851132392883,
407
- "learning_rate": 7.484565668918111e-05,
408
- "loss": 0.06109699010848999,
409
- "step": 285
410
- },
411
- {
412
- "epoch": 0.19781718963165076,
413
- "grad_norm": 1.4887421131134033,
414
- "learning_rate": 7.483622574790308e-05,
415
- "loss": 0.048966211080551145,
416
- "step": 290
417
- },
418
- {
419
- "epoch": 0.20122783083219645,
420
- "grad_norm": 0.5699282884597778,
421
- "learning_rate": 7.482651578449223e-05,
422
- "loss": 0.05427658557891846,
423
- "step": 295
424
- },
425
- {
426
- "epoch": 0.20463847203274216,
427
- "grad_norm": 1.6645292043685913,
428
- "learning_rate": 7.481652687151339e-05,
429
- "loss": 0.037466832995414735,
430
- "step": 300
431
- },
432
- {
433
- "epoch": 0.20804911323328787,
434
- "grad_norm": 0.4979431629180908,
435
- "learning_rate": 7.480625908361593e-05,
436
- "loss": 0.019084173440933227,
437
- "step": 305
438
- },
439
- {
440
- "epoch": 0.21145975443383355,
441
- "grad_norm": 2.73081636428833,
442
- "learning_rate": 7.479571249753339e-05,
443
- "loss": 0.07597044706344605,
444
- "step": 310
445
- },
446
- {
447
- "epoch": 0.21487039563437926,
448
- "grad_norm": 0.009097559377551079,
449
- "learning_rate": 7.478488719208281e-05,
450
- "loss": 0.017771795392036438,
451
- "step": 315
452
- },
453
- {
454
- "epoch": 0.21828103683492497,
455
- "grad_norm": 1.5284112691879272,
456
- "learning_rate": 7.477378324816419e-05,
457
- "loss": 0.07524526119232178,
458
- "step": 320
459
- },
460
- {
461
- "epoch": 0.22169167803547066,
462
- "grad_norm": 1.400959849357605,
463
- "learning_rate": 7.47624007487598e-05,
464
- "loss": 0.0357323557138443,
465
- "step": 325
466
- },
467
- {
468
- "epoch": 0.22510231923601637,
469
- "grad_norm": 0.5988397598266602,
470
- "learning_rate": 7.47507397789337e-05,
471
- "loss": 0.06072888970375061,
472
- "step": 330
473
- },
474
- {
475
- "epoch": 0.22851296043656208,
476
- "grad_norm": 0.18309183418750763,
477
- "learning_rate": 7.473880042583092e-05,
478
- "loss": 0.03904334008693695,
479
- "step": 335
480
- },
481
- {
482
- "epoch": 0.23192360163710776,
483
- "grad_norm": 0.7360084056854248,
484
- "learning_rate": 7.472658277867702e-05,
485
- "loss": 0.05045387148857117,
486
- "step": 340
487
- },
488
- {
489
- "epoch": 0.23533424283765347,
490
- "grad_norm": 2.315072536468506,
491
- "learning_rate": 7.471408692877724e-05,
492
- "loss": 0.07920202016830444,
493
- "step": 345
494
- },
495
- {
496
- "epoch": 0.23874488403819918,
497
- "grad_norm": 1.2811086177825928,
498
- "learning_rate": 7.470131296951592e-05,
499
- "loss": 0.05552580952644348,
500
- "step": 350
501
- },
502
- {
503
- "epoch": 0.2421555252387449,
504
- "grad_norm": 4.006563186645508,
505
- "learning_rate": 7.468826099635578e-05,
506
- "loss": 0.1419215679168701,
507
- "step": 355
508
- },
509
- {
510
- "epoch": 0.24556616643929058,
511
- "grad_norm": 1.1540688276290894,
512
- "learning_rate": 7.467493110683718e-05,
513
- "loss": 0.03980849981307984,
514
- "step": 360
515
- },
516
- {
517
- "epoch": 0.2489768076398363,
518
- "grad_norm": 1.5472272634506226,
519
- "learning_rate": 7.466132340057742e-05,
520
- "loss": 0.020862475037574768,
521
- "step": 365
522
- },
523
- {
524
- "epoch": 0.24965893587994542,
525
- "eval_loss": 0.11654457449913025,
526
- "eval_runtime": 1.0333,
527
- "eval_samples_per_second": 72.584,
528
- "eval_steps_per_second": 1.936,
529
- "step": 366
530
- },
531
- {
532
- "eval_cer_subset": 0.05232320479629377,
533
- "eval_cer_subset_edit_distance": 384,
534
- "eval_cer_subset_groups": 75,
535
- "eval_cer_subset_items": 75,
536
- "eval_cer_subset_ref_chars": 7339,
537
- "step": 366
538
- },
539
- {
540
- "epoch": 0.252387448840382,
541
- "grad_norm": 1.730627417564392,
542
- "learning_rate": 7.464743797927002e-05,
543
- "loss": 0.11239330768585205,
544
- "step": 370
545
- },
546
- {
547
- "epoch": 0.2557980900409277,
548
- "grad_norm": 0.1921682506799698,
549
- "learning_rate": 7.463327494668388e-05,
550
- "loss": 0.09260941743850708,
551
- "step": 375
552
- },
553
- {
554
- "epoch": 0.2592087312414734,
555
- "grad_norm": 1.9259151220321655,
556
- "learning_rate": 7.461883440866259e-05,
557
- "loss": 0.03999299705028534,
558
- "step": 380
559
- },
560
- {
561
- "epoch": 0.2626193724420191,
562
- "grad_norm": 0.0488249845802784,
563
- "learning_rate": 7.460411647312358e-05,
564
- "loss": 0.01459498256444931,
565
- "step": 385
566
- },
567
- {
568
- "epoch": 0.2660300136425648,
569
- "grad_norm": 1.1967735290527344,
570
- "learning_rate": 7.458912125005732e-05,
571
- "loss": 0.17716412544250487,
572
- "step": 390
573
- },
574
- {
575
- "epoch": 0.2694406548431105,
576
- "grad_norm": 1.0083205699920654,
577
- "learning_rate": 7.457384885152655e-05,
578
- "loss": 0.08738511800765991,
579
- "step": 395
580
- },
581
- {
582
- "epoch": 0.2728512960436562,
583
- "grad_norm": 0.6705593466758728,
584
- "learning_rate": 7.455829939166539e-05,
585
- "loss": 0.026945650577545166,
586
- "step": 400
587
- },
588
- {
589
- "epoch": 0.2762619372442019,
590
- "grad_norm": 1.0791361331939697,
591
- "learning_rate": 7.45424729866785e-05,
592
- "loss": 0.03804046213626862,
593
- "step": 405
594
- },
595
- {
596
- "epoch": 0.27967257844474763,
597
- "grad_norm": 0.7377910017967224,
598
- "learning_rate": 7.452636975484021e-05,
599
- "loss": 0.0464675635099411,
600
- "step": 410
601
- },
602
- {
603
- "epoch": 0.2830832196452933,
604
- "grad_norm": 0.8061625957489014,
605
- "learning_rate": 7.450998981649365e-05,
606
- "loss": 0.02737331986427307,
607
- "step": 415
608
- },
609
- {
610
- "epoch": 0.286493860845839,
611
- "grad_norm": 0.2580685019493103,
612
- "learning_rate": 7.449333329404982e-05,
613
- "loss": 0.018785761296749116,
614
- "step": 420
615
- },
616
- {
617
- "epoch": 0.28990450204638474,
618
- "grad_norm": 0.017052194103598595,
619
- "learning_rate": 7.447640031198675e-05,
620
- "loss": 0.11424320936203003,
621
- "step": 425
622
- },
623
- {
624
- "epoch": 0.2933151432469304,
625
- "grad_norm": 0.2855948805809021,
626
- "learning_rate": 7.445919099684845e-05,
627
- "loss": 0.012821969389915467,
628
- "step": 430
629
- },
630
- {
631
- "epoch": 0.2967257844474761,
632
- "grad_norm": 1.5070558786392212,
633
- "learning_rate": 7.444170547724405e-05,
634
- "loss": 0.037783479690551756,
635
- "step": 435
636
- },
637
- {
638
- "epoch": 0.30013642564802184,
639
- "grad_norm": 0.18241967260837555,
640
- "learning_rate": 7.442394388384684e-05,
641
- "loss": 0.03347713351249695,
642
- "step": 440
643
- },
644
- {
645
- "epoch": 0.3035470668485675,
646
- "grad_norm": 0.37319424748420715,
647
- "learning_rate": 7.440590634939327e-05,
648
- "loss": 0.05499382615089417,
649
- "step": 445
650
- },
651
- {
652
- "epoch": 0.3069577080491132,
653
- "grad_norm": 0.11083097755908966,
654
- "learning_rate": 7.438759300868193e-05,
655
- "loss": 0.021977408230304717,
656
- "step": 450
657
- },
658
- {
659
- "epoch": 0.31036834924965895,
660
- "grad_norm": 0.24985608458518982,
661
- "learning_rate": 7.436900399857261e-05,
662
- "loss": 0.07826730608940125,
663
- "step": 455
664
- },
665
- {
666
- "epoch": 0.31377899045020463,
667
- "grad_norm": 2.5360186100006104,
668
- "learning_rate": 7.43501394579852e-05,
669
- "loss": 0.080865478515625,
670
- "step": 460
671
- },
672
- {
673
- "epoch": 0.3171896316507503,
674
- "grad_norm": 0.7053658366203308,
675
- "learning_rate": 7.433099952789876e-05,
676
- "loss": 0.012464526295661926,
677
- "step": 465
678
- },
679
- {
680
- "epoch": 0.32060027285129605,
681
- "grad_norm": 0.3297032117843628,
682
- "learning_rate": 7.43115843513503e-05,
683
- "loss": 0.05623521208763123,
684
- "step": 470
685
- },
686
- {
687
- "epoch": 0.32401091405184174,
688
- "grad_norm": 1.3899471759796143,
689
- "learning_rate": 7.42918940734339e-05,
690
- "loss": 0.07306370735168458,
691
- "step": 475
692
- },
693
- {
694
- "epoch": 0.3274215552523875,
695
- "grad_norm": 0.9437380433082581,
696
- "learning_rate": 7.427192884129948e-05,
697
- "loss": 0.058290761709213254,
698
- "step": 480
699
- },
700
- {
701
- "epoch": 0.33083219645293316,
702
- "grad_norm": 1.8157323598861694,
703
- "learning_rate": 7.42516888041518e-05,
704
- "loss": 0.058532989025115965,
705
- "step": 485
706
- },
707
- {
708
- "epoch": 0.33424283765347884,
709
- "grad_norm": 0.6275774836540222,
710
- "learning_rate": 7.423117411324924e-05,
711
- "loss": 0.0624964714050293,
712
- "step": 490
713
- },
714
- {
715
- "epoch": 0.3376534788540246,
716
- "grad_norm": 0.6674565672874451,
717
- "learning_rate": 7.421038492190278e-05,
718
- "loss": 0.020014175772666933,
719
- "step": 495
720
- },
721
- {
722
- "epoch": 0.34106412005457026,
723
- "grad_norm": 0.6229842901229858,
724
- "learning_rate": 7.418932138547481e-05,
725
- "loss": 0.03286575376987457,
726
- "step": 500
727
- },
728
- {
729
- "epoch": 0.34447476125511595,
730
- "grad_norm": 1.441677212715149,
731
- "learning_rate": 7.41679836613779e-05,
732
- "loss": 0.04557921886444092,
733
- "step": 505
734
- },
735
- {
736
- "epoch": 0.3478854024556617,
737
- "grad_norm": 0.8439592719078064,
738
- "learning_rate": 7.414637190907379e-05,
739
- "loss": 0.027792316675186158,
740
- "step": 510
741
- },
742
- {
743
- "epoch": 0.35129604365620737,
744
- "grad_norm": 0.1357346475124359,
745
- "learning_rate": 7.412448629007198e-05,
746
- "loss": 0.024153730273246764,
747
- "step": 515
748
- },
749
- {
750
- "epoch": 0.35470668485675305,
751
- "grad_norm": 0.11876281350851059,
752
- "learning_rate": 7.41023269679287e-05,
753
- "loss": 0.11651531457901002,
754
- "step": 520
755
- },
756
- {
757
- "epoch": 0.3581173260572988,
758
- "grad_norm": 0.8576210737228394,
759
- "learning_rate": 7.407989410824566e-05,
760
- "loss": 0.045156928896903994,
761
- "step": 525
762
- },
763
- {
764
- "epoch": 0.3615279672578445,
765
- "grad_norm": 0.39947113394737244,
766
- "learning_rate": 7.40571878786687e-05,
767
- "loss": 0.02562606930732727,
768
- "step": 530
769
- },
770
- {
771
- "epoch": 0.36493860845839016,
772
- "grad_norm": 0.8822716474533081,
773
- "learning_rate": 7.403420844888668e-05,
774
- "loss": 0.05394383668899536,
775
- "step": 535
776
- },
777
- {
778
- "epoch": 0.3683492496589359,
779
- "grad_norm": 1.8026832342147827,
780
- "learning_rate": 7.40109559906301e-05,
781
- "loss": 0.07706952095031738,
782
- "step": 540
783
- },
784
- {
785
- "epoch": 0.3717598908594816,
786
- "grad_norm": 0.28902706503868103,
787
- "learning_rate": 7.398743067766987e-05,
788
- "loss": 0.0352792352437973,
789
- "step": 545
790
- },
791
- {
792
- "epoch": 0.37517053206002726,
793
- "grad_norm": 0.2759908437728882,
794
- "learning_rate": 7.396363268581609e-05,
795
- "loss": 0.038266700506210324,
796
- "step": 550
797
- },
798
- {
799
- "epoch": 0.378581173260573,
800
- "grad_norm": 1.0520153045654297,
801
- "learning_rate": 7.39395621929165e-05,
802
- "loss": 0.04206843376159668,
803
- "step": 555
804
- },
805
- {
806
- "epoch": 0.3819918144611187,
807
- "grad_norm": 0.29290419816970825,
808
- "learning_rate": 7.391521937885543e-05,
809
- "loss": 0.04060278534889221,
810
- "step": 560
811
- },
812
- {
813
- "epoch": 0.38540245566166437,
814
- "grad_norm": 0.11243477463722229,
815
- "learning_rate": 7.389060442555228e-05,
816
- "loss": 0.05412468910217285,
817
- "step": 565
818
- },
819
- {
820
- "epoch": 0.3888130968622101,
821
- "grad_norm": 1.9416879415512085,
822
- "learning_rate": 7.386571751696019e-05,
823
- "loss": 0.02231921851634979,
824
- "step": 570
825
- },
826
- {
827
- "epoch": 0.3922237380627558,
828
- "grad_norm": 0.5937671661376953,
829
- "learning_rate": 7.384055883906474e-05,
830
- "loss": 0.032561862468719484,
831
- "step": 575
832
- },
833
- {
834
- "epoch": 0.3956343792633015,
835
- "grad_norm": 0.026148535311222076,
836
- "learning_rate": 7.381512857988244e-05,
837
- "loss": 0.07647547125816345,
838
- "step": 580
839
- },
840
- {
841
- "epoch": 0.3990450204638472,
842
- "grad_norm": 0.7875772714614868,
843
- "learning_rate": 7.378942692945944e-05,
844
- "loss": 0.031203645467758178,
845
- "step": 585
846
- },
847
- {
848
- "epoch": 0.4024556616643929,
849
- "grad_norm": 0.45512810349464417,
850
- "learning_rate": 7.376345407987002e-05,
851
- "loss": 0.04238590002059937,
852
- "step": 590
853
- },
854
- {
855
- "epoch": 0.40586630286493863,
856
- "grad_norm": 1.66355299949646,
857
- "learning_rate": 7.373721022521521e-05,
858
- "loss": 0.052533066272735594,
859
- "step": 595
860
- },
861
- {
862
- "epoch": 0.4092769440654843,
863
- "grad_norm": 0.08107655495405197,
864
- "learning_rate": 7.371069556162133e-05,
865
- "loss": 0.017715978622436523,
866
- "step": 600
867
- },
868
- {
869
- "epoch": 0.41268758526603,
870
- "grad_norm": 0.32274800539016724,
871
- "learning_rate": 7.368391028723851e-05,
872
- "loss": 0.1379294991493225,
873
- "step": 605
874
- },
875
- {
876
- "epoch": 0.41609822646657574,
877
- "grad_norm": 1.8197475671768188,
878
- "learning_rate": 7.365685460223922e-05,
879
- "loss": 0.03312918543815613,
880
- "step": 610
881
- },
882
- {
883
- "epoch": 0.4195088676671214,
884
- "grad_norm": 0.1390945166349411,
885
- "learning_rate": 7.362952870881677e-05,
886
- "loss": 0.027584537863731384,
887
- "step": 615
888
- },
889
- {
890
- "epoch": 0.4229195088676671,
891
- "grad_norm": 0.9081276655197144,
892
- "learning_rate": 7.360193281118378e-05,
893
- "loss": 0.06143233776092529,
894
- "step": 620
895
- },
896
- {
897
- "epoch": 0.42633015006821284,
898
- "grad_norm": 0.07777975499629974,
899
- "learning_rate": 7.35740671155707e-05,
900
- "loss": 0.053598570823669436,
901
- "step": 625
902
- },
903
- {
904
- "epoch": 0.4297407912687585,
905
- "grad_norm": 0.9314269423484802,
906
- "learning_rate": 7.354593183022422e-05,
907
- "loss": 0.05946495532989502,
908
- "step": 630
909
- },
910
- {
911
- "epoch": 0.4331514324693042,
912
- "grad_norm": 0.5312000513076782,
913
- "learning_rate": 7.351752716540575e-05,
914
- "loss": 0.030707958340644836,
915
- "step": 635
916
- },
917
- {
918
- "epoch": 0.43656207366984995,
919
- "grad_norm": 0.855117917060852,
920
- "learning_rate": 7.348885333338984e-05,
921
- "loss": 0.09321808815002441,
922
- "step": 640
923
- },
924
- {
925
- "epoch": 0.43997271487039563,
926
- "grad_norm": 0.12914253771305084,
927
- "learning_rate": 7.345991054846257e-05,
928
- "loss": 0.010356919467449188,
929
- "step": 645
930
- },
931
- {
932
- "epoch": 0.4433833560709413,
933
- "grad_norm": 0.4129096567630768,
934
- "learning_rate": 7.343069902691999e-05,
935
- "loss": 0.054264682531356814,
936
- "step": 650
937
- },
938
- {
939
- "epoch": 0.44679399727148705,
940
- "grad_norm": 1.8499324321746826,
941
- "learning_rate": 7.340121898706643e-05,
942
- "loss": 0.050659948587417604,
943
- "step": 655
944
- },
945
- {
946
- "epoch": 0.45020463847203274,
947
- "grad_norm": 0.5490806698799133,
948
- "learning_rate": 7.337147064921299e-05,
949
- "loss": 0.07158003449440002,
950
- "step": 660
951
- },
952
- {
953
- "epoch": 0.4536152796725784,
954
- "grad_norm": 1.1408376693725586,
955
- "learning_rate": 7.334145423567575e-05,
956
- "loss": 0.08845412135124206,
957
- "step": 665
958
- },
959
- {
960
- "epoch": 0.45702592087312416,
961
- "grad_norm": 1.5242546796798706,
962
- "learning_rate": 7.331116997077426e-05,
963
- "loss": 0.07773985266685486,
964
- "step": 670
965
- },
966
- {
967
- "epoch": 0.46043656207366984,
968
- "grad_norm": 0.7061560153961182,
969
- "learning_rate": 7.32806180808297e-05,
970
- "loss": 0.047228410840034485,
971
- "step": 675
972
- },
973
- {
974
- "epoch": 0.4638472032742155,
975
- "grad_norm": 0.8088539838790894,
976
- "learning_rate": 7.324979879416333e-05,
977
- "loss": 0.03726888597011566,
978
- "step": 680
979
- },
980
- {
981
- "epoch": 0.46725784447476126,
982
- "grad_norm": 0.5670620799064636,
983
- "learning_rate": 7.321871234109472e-05,
984
- "loss": 0.02899191677570343,
985
- "step": 685
986
- },
987
- {
988
- "epoch": 0.47066848567530695,
989
- "grad_norm": 2.3821427822113037,
990
- "learning_rate": 7.318735895394e-05,
991
- "loss": 0.033483856916427614,
992
- "step": 690
993
- },
994
- {
995
- "epoch": 0.4740791268758527,
996
- "grad_norm": 0.8073883652687073,
997
- "learning_rate": 7.315573886701023e-05,
998
- "loss": 0.05756385326385498,
999
- "step": 695
1000
- },
1001
- {
1002
- "epoch": 0.47748976807639837,
1003
- "grad_norm": 0.09120920300483704,
1004
- "learning_rate": 7.31238523166095e-05,
1005
- "loss": 0.0338085800409317,
1006
- "step": 700
1007
- },
1008
- {
1009
- "epoch": 0.48090040927694405,
1010
- "grad_norm": 0.33443862199783325,
1011
- "learning_rate": 7.309169954103326e-05,
1012
- "loss": 0.00844155102968216,
1013
- "step": 705
1014
- },
1015
- {
1016
- "epoch": 0.4843110504774898,
1017
- "grad_norm": 0.4880702793598175,
1018
- "learning_rate": 7.305928078056657e-05,
1019
- "loss": 0.09532383680343628,
1020
- "step": 710
1021
- },
1022
- {
1023
- "epoch": 0.4877216916780355,
1024
- "grad_norm": 0.11862733215093613,
1025
- "learning_rate": 7.302659627748221e-05,
1026
- "loss": 0.01845739334821701,
1027
- "step": 715
1028
- },
1029
- {
1030
- "epoch": 0.49113233287858116,
1031
- "grad_norm": 0.03655651956796646,
1032
- "learning_rate": 7.299364627603892e-05,
1033
- "loss": 0.030477851629257202,
1034
- "step": 720
1035
- },
1036
- {
1037
- "epoch": 0.4945429740791269,
1038
- "grad_norm": 1.481441617012024,
1039
- "learning_rate": 7.29604310224796e-05,
1040
- "loss": 0.07586092352867127,
1041
- "step": 725
1042
- },
1043
- {
1044
- "epoch": 0.4979536152796726,
1045
- "grad_norm": 0.8510580658912659,
1046
- "learning_rate": 7.292695076502938e-05,
1047
- "loss": 0.03589251637458801,
1048
- "step": 730
1049
- },
1050
- {
1051
- "epoch": 0.49931787175989084,
1052
- "eval_loss": 0.061700768768787384,
1053
- "eval_runtime": 0.8886,
1054
- "eval_samples_per_second": 84.399,
1055
- "eval_steps_per_second": 2.251,
1056
- "step": 732
1057
- },
1058
- {
1059
- "eval_cer_subset": 0.021256301948494344,
1060
- "eval_cer_subset_edit_distance": 156,
1061
- "eval_cer_subset_groups": 75,
1062
- "eval_cer_subset_items": 75,
1063
- "eval_cer_subset_ref_chars": 7339,
1064
- "step": 732
1065
- },
1066
- {
1067
- "epoch": 0.5013642564802183,
1068
- "grad_norm": 0.49610504508018494,
1069
- "learning_rate": 7.28932057538939e-05,
1070
- "loss": 0.06440846920013428,
1071
- "step": 735
1072
- },
1073
- {
1074
- "epoch": 0.504774897680764,
1075
- "grad_norm": 0.5659550428390503,
1076
- "learning_rate": 7.285919624125732e-05,
1077
- "loss": 0.08426347374916077,
1078
- "step": 740
1079
- },
1080
- {
1081
- "epoch": 0.5081855388813097,
1082
- "grad_norm": 0.04723689705133438,
1083
- "learning_rate": 7.282492248128047e-05,
1084
- "loss": 0.07788341641426086,
1085
- "step": 745
1086
- },
1087
- {
1088
- "epoch": 0.5115961800818554,
1089
- "grad_norm": 0.720941960811615,
1090
- "learning_rate": 7.2790384730099e-05,
1091
- "loss": 0.03100808262825012,
1092
- "step": 750
1093
- },
1094
- {
1095
- "epoch": 0.515006821282401,
1096
- "grad_norm": 0.652988851070404,
1097
- "learning_rate": 7.275558324582138e-05,
1098
- "loss": 0.03954651951789856,
1099
- "step": 755
1100
- },
1101
- {
1102
- "epoch": 0.5184174624829468,
1103
- "grad_norm": 1.2214330434799194,
1104
- "learning_rate": 7.272051828852705e-05,
1105
- "loss": 0.019992084801197053,
1106
- "step": 760
1107
- },
1108
- {
1109
- "epoch": 0.5218281036834925,
1110
- "grad_norm": 1.292953372001648,
1111
- "learning_rate": 7.268519012026443e-05,
1112
- "loss": 0.07394988536834717,
1113
- "step": 765
1114
- },
1115
- {
1116
- "epoch": 0.5252387448840382,
1117
- "grad_norm": 1.1823278665542603,
1118
- "learning_rate": 7.264959900504901e-05,
1119
- "loss": 0.037449967861175534,
1120
- "step": 770
1121
- },
1122
- {
1123
- "epoch": 0.5286493860845839,
1124
- "grad_norm": 1.1314970254898071,
1125
- "learning_rate": 7.261374520886128e-05,
1126
- "loss": 0.04381995797157288,
1127
- "step": 775
1128
- },
1129
- {
1130
- "epoch": 0.5320600272851296,
1131
- "grad_norm": 0.02543286792933941,
1132
- "learning_rate": 7.257762899964486e-05,
1133
- "loss": 0.07130052447319031,
1134
- "step": 780
1135
- },
1136
- {
1137
- "epoch": 0.5354706684856753,
1138
- "grad_norm": 0.37440457940101624,
1139
- "learning_rate": 7.25412506473044e-05,
1140
- "loss": 0.050841158628463744,
1141
- "step": 785
1142
- },
1143
- {
1144
- "epoch": 0.538881309686221,
1145
- "grad_norm": 0.2532084882259369,
1146
- "learning_rate": 7.250461042370365e-05,
1147
- "loss": 0.03486245274543762,
1148
- "step": 790
1149
- },
1150
- {
1151
- "epoch": 0.5422919508867667,
1152
- "grad_norm": 1.0150209665298462,
1153
- "learning_rate": 7.246770860266333e-05,
1154
- "loss": 0.050749993324279784,
1155
- "step": 795
1156
- },
1157
- {
1158
- "epoch": 0.5457025920873124,
1159
- "grad_norm": 1.108716607093811,
1160
- "learning_rate": 7.24305454599592e-05,
1161
- "loss": 0.03166365325450897,
1162
- "step": 800
1163
- },
1164
- {
1165
- "epoch": 0.5491132332878581,
1166
- "grad_norm": 0.16657976806163788,
1167
- "learning_rate": 7.239312127331989e-05,
1168
- "loss": 0.05016656517982483,
1169
- "step": 805
1170
- },
1171
- {
1172
- "epoch": 0.5525238744884038,
1173
- "grad_norm": 0.005627671722322702,
1174
- "learning_rate": 7.235543632242488e-05,
1175
- "loss": 0.021701858937740327,
1176
- "step": 810
1177
- },
1178
- {
1179
- "epoch": 0.5559345156889495,
1180
- "grad_norm": 1.8796381950378418,
1181
- "learning_rate": 7.231749088890241e-05,
1182
- "loss": 0.061094462871551514,
1183
- "step": 815
1184
- },
1185
- {
1186
- "epoch": 0.5593451568894953,
1187
- "grad_norm": 0.020010950043797493,
1188
- "learning_rate": 7.227928525632737e-05,
1189
- "loss": 0.0238655224442482,
1190
- "step": 820
1191
- },
1192
- {
1193
- "epoch": 0.562755798090041,
1194
- "grad_norm": 1.9777793884277344,
1195
- "learning_rate": 7.224081971021914e-05,
1196
- "loss": 0.041665592789649965,
1197
- "step": 825
1198
- },
1199
- {
1200
- "epoch": 0.5661664392905866,
1201
- "grad_norm": 0.06644955277442932,
1202
- "learning_rate": 7.220209453803954e-05,
1203
- "loss": 0.016651667654514313,
1204
- "step": 830
1205
- },
1206
- {
1207
- "epoch": 0.5695770804911323,
1208
- "grad_norm": 0.8203716278076172,
1209
- "learning_rate": 7.216311002919064e-05,
1210
- "loss": 0.03519523441791535,
1211
- "step": 835
1212
- },
1213
- {
1214
- "epoch": 0.572987721691678,
1215
- "grad_norm": 1.957996129989624,
1216
- "learning_rate": 7.212386647501254e-05,
1217
- "loss": 0.03704521656036377,
1218
- "step": 840
1219
- },
1220
- {
1221
- "epoch": 0.5763983628922238,
1222
- "grad_norm": 0.2386065572500229,
1223
- "learning_rate": 7.208436416878125e-05,
1224
- "loss": 0.02845575213432312,
1225
- "step": 845
1226
- },
1227
- {
1228
- "epoch": 0.5798090040927695,
1229
- "grad_norm": 0.9101418256759644,
1230
- "learning_rate": 7.204460340570658e-05,
1231
- "loss": 0.01103741154074669,
1232
- "step": 850
1233
- },
1234
- {
1235
- "epoch": 0.5832196452933152,
1236
- "grad_norm": 0.22701004147529602,
1237
- "learning_rate": 7.200458448292972e-05,
1238
- "loss": 0.06184377670288086,
1239
- "step": 855
1240
- },
1241
- {
1242
- "epoch": 0.5866302864938608,
1243
- "grad_norm": 2.3091611862182617,
1244
- "learning_rate": 7.196430769952126e-05,
1245
- "loss": 0.0492431253194809,
1246
- "step": 860
1247
- },
1248
- {
1249
- "epoch": 0.5900409276944065,
1250
- "grad_norm": 0.6630973815917969,
1251
- "learning_rate": 7.192377335647876e-05,
1252
- "loss": 0.027876955270767213,
1253
- "step": 865
1254
- },
1255
- {
1256
- "epoch": 0.5934515688949522,
1257
- "grad_norm": 1.7400578260421753,
1258
- "learning_rate": 7.188298175672464e-05,
1259
- "loss": 0.023742210865020753,
1260
- "step": 870
1261
- },
1262
- {
1263
- "epoch": 0.596862210095498,
1264
- "grad_norm": 0.7121092081069946,
1265
- "learning_rate": 7.184193320510379e-05,
1266
- "loss": 0.02125793993473053,
1267
- "step": 875
1268
- },
1269
- {
1270
- "epoch": 0.6002728512960437,
1271
- "grad_norm": 0.429611474275589,
1272
- "learning_rate": 7.180062800838143e-05,
1273
- "loss": 0.06682519316673279,
1274
- "step": 880
1275
- },
1276
- {
1277
- "epoch": 0.6036834924965894,
1278
- "grad_norm": 0.018405891954898834,
1279
- "learning_rate": 7.17590664752407e-05,
1280
- "loss": 0.022519922256469725,
1281
- "step": 885
1282
- },
1283
- {
1284
- "epoch": 0.607094133697135,
1285
- "grad_norm": 0.9797202348709106,
1286
- "learning_rate": 7.171724891628046e-05,
1287
- "loss": 0.10513803958892823,
1288
- "step": 890
1289
- },
1290
- {
1291
- "epoch": 0.6105047748976807,
1292
- "grad_norm": 0.11931606382131577,
1293
- "learning_rate": 7.167517564401282e-05,
1294
- "loss": 0.055521953105926516,
1295
- "step": 895
1296
- },
1297
- {
1298
- "epoch": 0.6139154160982264,
1299
- "grad_norm": 0.04398813843727112,
1300
- "learning_rate": 7.163284697286097e-05,
1301
- "loss": 0.018342888355255126,
1302
- "step": 900
1303
- },
1304
- {
1305
- "epoch": 0.6173260572987722,
1306
- "grad_norm": 0.11505168676376343,
1307
- "learning_rate": 7.15902632191567e-05,
1308
- "loss": 0.026946401596069335,
1309
- "step": 905
1310
- },
1311
- {
1312
- "epoch": 0.6207366984993179,
1313
- "grad_norm": 0.3042922019958496,
1314
- "learning_rate": 7.154742470113816e-05,
1315
- "loss": 0.02314314842224121,
1316
- "step": 910
1317
- },
1318
- {
1319
- "epoch": 0.6241473396998636,
1320
- "grad_norm": 0.09922346472740173,
1321
- "learning_rate": 7.150433173894733e-05,
1322
- "loss": 0.06378543972969056,
1323
- "step": 915
1324
- },
1325
- {
1326
- "epoch": 0.6275579809004093,
1327
- "grad_norm": 0.6513009667396545,
1328
- "learning_rate": 7.146098465462776e-05,
1329
- "loss": 0.036993378400802614,
1330
- "step": 920
1331
- },
1332
- {
1333
- "epoch": 0.630968622100955,
1334
- "grad_norm": 1.131602168083191,
1335
- "learning_rate": 7.14173837721221e-05,
1336
- "loss": 0.09491010308265686,
1337
- "step": 925
1338
- },
1339
- {
1340
- "epoch": 0.6343792633015006,
1341
- "grad_norm": 0.1672857254743576,
1342
- "learning_rate": 7.137352941726969e-05,
1343
- "loss": 0.03719751834869385,
1344
- "step": 930
1345
- },
1346
- {
1347
- "epoch": 0.6377899045020464,
1348
- "grad_norm": 0.7450887560844421,
1349
- "learning_rate": 7.132942191780414e-05,
1350
- "loss": 0.028598809242248537,
1351
- "step": 935
1352
- },
1353
- {
1354
- "epoch": 0.6412005457025921,
1355
- "grad_norm": 2.3537657260894775,
1356
- "learning_rate": 7.128506160335084e-05,
1357
- "loss": 0.06678735613822936,
1358
- "step": 940
1359
- },
1360
- {
1361
- "epoch": 0.6446111869031378,
1362
- "grad_norm": 0.014428210444748402,
1363
- "learning_rate": 7.124044880542455e-05,
1364
- "loss": 0.018354634940624236,
1365
- "step": 945
1366
- },
1367
- {
1368
- "epoch": 0.6480218281036835,
1369
- "grad_norm": 2.9239041805267334,
1370
- "learning_rate": 7.119558385742688e-05,
1371
- "loss": 0.08242651224136352,
1372
- "step": 950
1373
- },
1374
- {
1375
- "epoch": 0.6514324693042292,
1376
- "grad_norm": 0.39032718539237976,
1377
- "learning_rate": 7.115046709464383e-05,
1378
- "loss": 0.023772728443145753,
1379
- "step": 955
1380
- },
1381
- {
1382
- "epoch": 0.654843110504775,
1383
- "grad_norm": 0.3000798523426056,
1384
- "learning_rate": 7.110509885424326e-05,
1385
- "loss": 0.03464276790618896,
1386
- "step": 960
1387
- },
1388
- {
1389
- "epoch": 0.6582537517053206,
1390
- "grad_norm": 0.3980049192905426,
1391
- "learning_rate": 7.105947947527238e-05,
1392
- "loss": 0.08540127277374268,
1393
- "step": 965
1394
- },
1395
- {
1396
- "epoch": 0.6616643929058663,
1397
- "grad_norm": 0.9492272734642029,
1398
- "learning_rate": 7.10136092986552e-05,
1399
- "loss": 0.02300785481929779,
1400
- "step": 970
1401
- },
1402
- {
1403
- "epoch": 0.665075034106412,
1404
- "grad_norm": 1.9585710763931274,
1405
- "learning_rate": 7.096748866719005e-05,
1406
- "loss": 0.034704044461250305,
1407
- "step": 975
1408
- },
1409
- {
1410
- "epoch": 0.6684856753069577,
1411
- "grad_norm": 1.142238974571228,
1412
- "learning_rate": 7.092111792554689e-05,
1413
- "loss": 0.01860647052526474,
1414
- "step": 980
1415
- },
1416
- {
1417
- "epoch": 0.6718963165075034,
1418
- "grad_norm": 0.8443534970283508,
1419
- "learning_rate": 7.087449742026488e-05,
1420
- "loss": 0.03302992284297943,
1421
- "step": 985
1422
- },
1423
- {
1424
- "epoch": 0.6753069577080492,
1425
- "grad_norm": 1.0402863025665283,
1426
- "learning_rate": 7.082762749974968e-05,
1427
- "loss": 0.03963000178337097,
1428
- "step": 990
1429
- },
1430
- {
1431
- "epoch": 0.6787175989085948,
1432
- "grad_norm": 0.11959892511367798,
1433
- "learning_rate": 7.078050851427089e-05,
1434
- "loss": 0.0187692254781723,
1435
- "step": 995
1436
- },
1437
- {
1438
- "epoch": 0.6821282401091405,
1439
- "grad_norm": 1.495011329650879,
1440
- "learning_rate": 7.073314081595945e-05,
1441
- "loss": 0.050608736276626584,
1442
- "step": 1000
1443
- },
1444
- {
1445
- "epoch": 0.6855388813096862,
1446
- "grad_norm": 0.2534504234790802,
1447
- "learning_rate": 7.068552475880499e-05,
1448
- "loss": 0.0076520174741745,
1449
- "step": 1005
1450
- },
1451
- {
1452
- "epoch": 0.6889495225102319,
1453
- "grad_norm": 0.17387191951274872,
1454
- "learning_rate": 7.063766069865314e-05,
1455
- "loss": 0.028283193707466125,
1456
- "step": 1010
1457
- },
1458
- {
1459
- "epoch": 0.6923601637107776,
1460
- "grad_norm": 0.07424458116292953,
1461
- "learning_rate": 7.058954899320297e-05,
1462
- "loss": 0.03078552782535553,
1463
- "step": 1015
1464
- },
1465
- {
1466
- "epoch": 0.6957708049113234,
1467
- "grad_norm": 0.5854848027229309,
1468
- "learning_rate": 7.05411900020042e-05,
1469
- "loss": 0.02033105492591858,
1470
- "step": 1020
1471
- },
1472
- {
1473
- "epoch": 0.699181446111869,
1474
- "grad_norm": 0.09108871966600418,
1475
- "learning_rate": 7.049258408645463e-05,
1476
- "loss": 0.0510578989982605,
1477
- "step": 1025
1478
- },
1479
- {
1480
- "epoch": 0.7025920873124147,
1481
- "grad_norm": 0.2851751148700714,
1482
- "learning_rate": 7.044373160979734e-05,
1483
- "loss": 0.10413439273834228,
1484
- "step": 1030
1485
- },
1486
- {
1487
- "epoch": 0.7060027285129604,
1488
- "grad_norm": 0.8032590746879578,
1489
- "learning_rate": 7.039463293711804e-05,
1490
- "loss": 0.05853385329246521,
1491
- "step": 1035
1492
- },
1493
- {
1494
- "epoch": 0.7094133697135061,
1495
- "grad_norm": 0.1301775723695755,
1496
- "learning_rate": 7.03452884353423e-05,
1497
- "loss": 0.051472657918930055,
1498
- "step": 1040
1499
- },
1500
- {
1501
- "epoch": 0.7128240109140518,
1502
- "grad_norm": 0.46156957745552063,
1503
- "learning_rate": 7.029569847323287e-05,
1504
- "loss": 0.034115567803382874,
1505
- "step": 1045
1506
- },
1507
- {
1508
- "epoch": 0.7162346521145976,
1509
- "grad_norm": 1.081560730934143,
1510
- "learning_rate": 7.02458634213868e-05,
1511
- "loss": 0.059652507305145264,
1512
- "step": 1050
1513
- },
1514
- {
1515
- "epoch": 0.7196452933151433,
1516
- "grad_norm": 0.721208930015564,
1517
- "learning_rate": 7.019578365223286e-05,
1518
- "loss": 0.061070340871810916,
1519
- "step": 1055
1520
- },
1521
- {
1522
- "epoch": 0.723055934515689,
1523
- "grad_norm": 0.5738947987556458,
1524
- "learning_rate": 7.014545954002855e-05,
1525
- "loss": 0.03556577265262604,
1526
- "step": 1060
1527
- },
1528
- {
1529
- "epoch": 0.7264665757162346,
1530
- "grad_norm": 0.15053460001945496,
1531
- "learning_rate": 7.009489146085744e-05,
1532
- "loss": 0.03284372091293335,
1533
- "step": 1065
1534
- },
1535
- {
1536
- "epoch": 0.7298772169167803,
1537
- "grad_norm": 0.4496553838253021,
1538
- "learning_rate": 7.004407979262635e-05,
1539
- "loss": 0.07945018410682678,
1540
- "step": 1070
1541
- },
1542
- {
1543
- "epoch": 0.7332878581173261,
1544
- "grad_norm": 1.1821213960647583,
1545
- "learning_rate": 6.999302491506245e-05,
1546
- "loss": 0.033741748332977294,
1547
- "step": 1075
1548
- },
1549
- {
1550
- "epoch": 0.7366984993178718,
1551
- "grad_norm": 0.2809429168701172,
1552
- "learning_rate": 6.994172720971047e-05,
1553
- "loss": 0.023005199432373048,
1554
- "step": 1080
1555
- },
1556
- {
1557
- "epoch": 0.7401091405184175,
1558
- "grad_norm": 0.14925819635391235,
1559
- "learning_rate": 6.989018705992991e-05,
1560
- "loss": 0.01791207939386368,
1561
- "step": 1085
1562
- },
1563
- {
1564
- "epoch": 0.7435197817189632,
1565
- "grad_norm": 1.1947131156921387,
1566
- "learning_rate": 6.983840485089203e-05,
1567
- "loss": 0.03395574688911438,
1568
- "step": 1090
1569
- },
1570
- {
1571
- "epoch": 0.7469304229195088,
1572
- "grad_norm": 1.336547613143921,
1573
- "learning_rate": 6.978638096957712e-05,
1574
- "loss": 0.02712726593017578,
1575
- "step": 1095
1576
- },
1577
- {
1578
- "epoch": 0.7489768076398363,
1579
- "eval_loss": 0.05635881423950195,
1580
- "eval_runtime": 0.8951,
1581
- "eval_samples_per_second": 83.789,
1582
- "eval_steps_per_second": 2.234,
1583
- "step": 1098
1584
- },
1585
- {
1586
- "eval_cer_subset": 0.023981468864967978,
1587
- "eval_cer_subset_edit_distance": 176,
1588
- "eval_cer_subset_groups": 75,
1589
- "eval_cer_subset_items": 75,
1590
- "eval_cer_subset_ref_chars": 7339,
1591
- "step": 1098
1592
- },
1593
- {
1594
- "epoch": 0.7503410641200545,
1595
- "grad_norm": 0.015995435416698456,
1596
- "learning_rate": 6.973411580477149e-05,
1597
- "loss": 0.018527305126190184,
1598
- "step": 1100
1599
- },
1600
- {
1601
- "epoch": 0.7537517053206003,
1602
- "grad_norm": 2.3465819358825684,
1603
- "learning_rate": 6.968160974706465e-05,
1604
- "loss": 0.03113352656364441,
1605
- "step": 1105
1606
- },
1607
- {
1608
- "epoch": 0.757162346521146,
1609
- "grad_norm": 8.048991203308105,
1610
- "learning_rate": 6.962886318884633e-05,
1611
- "loss": 0.01905607581138611,
1612
- "step": 1110
1613
- },
1614
- {
1615
- "epoch": 0.7605729877216917,
1616
- "grad_norm": 2.0705132484436035,
1617
- "learning_rate": 6.957587652430363e-05,
1618
- "loss": 0.08121066093444824,
1619
- "step": 1115
1620
- },
1621
- {
1622
- "epoch": 0.7639836289222374,
1623
- "grad_norm": 0.6205260157585144,
1624
- "learning_rate": 6.952265014941796e-05,
1625
- "loss": 0.030836066603660582,
1626
- "step": 1120
1627
- },
1628
- {
1629
- "epoch": 0.767394270122783,
1630
- "grad_norm": 0.02030811458826065,
1631
- "learning_rate": 6.946918446196215e-05,
1632
- "loss": 0.0715795874595642,
1633
- "step": 1125
1634
- },
1635
- {
1636
- "epoch": 0.7708049113233287,
1637
- "grad_norm": 0.20006906986236572,
1638
- "learning_rate": 6.94154798614975e-05,
1639
- "loss": 0.09267728328704834,
1640
- "step": 1130
1641
- },
1642
- {
1643
- "epoch": 0.7742155525238745,
1644
- "grad_norm": 1.3217955827713013,
1645
- "learning_rate": 6.936153674937074e-05,
1646
- "loss": 0.057087546586990355,
1647
- "step": 1135
1648
- },
1649
- {
1650
- "epoch": 0.7776261937244202,
1651
- "grad_norm": 0.97421795129776,
1652
- "learning_rate": 6.930735552871105e-05,
1653
- "loss": 0.02381356656551361,
1654
- "step": 1140
1655
- },
1656
- {
1657
- "epoch": 0.7810368349249659,
1658
- "grad_norm": 1.1994248628616333,
1659
- "learning_rate": 6.925293660442705e-05,
1660
- "loss": 0.03957775831222534,
1661
- "step": 1145
1662
- },
1663
- {
1664
- "epoch": 0.7844474761255116,
1665
- "grad_norm": 1.0654077529907227,
1666
- "learning_rate": 6.919828038320378e-05,
1667
- "loss": 0.04088171124458313,
1668
- "step": 1150
1669
- },
1670
- {
1671
- "epoch": 0.7878581173260573,
1672
- "grad_norm": 0.6241940855979919,
1673
- "learning_rate": 6.914338727349963e-05,
1674
- "loss": 0.0698228895664215,
1675
- "step": 1155
1676
- },
1677
- {
1678
- "epoch": 0.791268758526603,
1679
- "grad_norm": 1.4655344486236572,
1680
- "learning_rate": 6.908825768554337e-05,
1681
- "loss": 0.0430897206068039,
1682
- "step": 1160
1683
- },
1684
- {
1685
- "epoch": 0.7946793997271487,
1686
- "grad_norm": 0.3493589162826538,
1687
- "learning_rate": 6.903289203133096e-05,
1688
- "loss": 0.05850836634635925,
1689
- "step": 1165
1690
- },
1691
- {
1692
- "epoch": 0.7980900409276944,
1693
- "grad_norm": 1.1164323091506958,
1694
- "learning_rate": 6.897729072462257e-05,
1695
- "loss": 0.02702825963497162,
1696
- "step": 1170
1697
- },
1698
- {
1699
- "epoch": 0.8015006821282401,
1700
- "grad_norm": 0.056947700679302216,
1701
- "learning_rate": 6.892145418093947e-05,
1702
- "loss": 0.11043190956115723,
1703
- "step": 1175
1704
- },
1705
- {
1706
- "epoch": 0.8049113233287858,
1707
- "grad_norm": 1.2450393438339233,
1708
- "learning_rate": 6.886538281756085e-05,
1709
- "loss": 0.06005706787109375,
1710
- "step": 1180
1711
- },
1712
- {
1713
- "epoch": 0.8083219645293315,
1714
- "grad_norm": 0.10885969549417496,
1715
- "learning_rate": 6.880907705352083e-05,
1716
- "loss": 0.022018623352050782,
1717
- "step": 1185
1718
- },
1719
- {
1720
- "epoch": 0.8117326057298773,
1721
- "grad_norm": 0.17044247686862946,
1722
- "learning_rate": 6.875253730960522e-05,
1723
- "loss": 0.024727573990821837,
1724
- "step": 1190
1725
- },
1726
- {
1727
- "epoch": 0.815143246930423,
1728
- "grad_norm": 0.22665634751319885,
1729
- "learning_rate": 6.869576400834843e-05,
1730
- "loss": 0.014500510692596436,
1731
- "step": 1195
1732
- },
1733
- {
1734
- "epoch": 0.8185538881309686,
1735
- "grad_norm": 0.6808337569236755,
1736
- "learning_rate": 6.863875757403028e-05,
1737
- "loss": 0.040313297510147096,
1738
- "step": 1200
1739
- },
1740
- {
1741
- "epoch": 0.8219645293315143,
1742
- "grad_norm": 0.37714090943336487,
1743
- "learning_rate": 6.858151843267289e-05,
1744
- "loss": 0.02225676029920578,
1745
- "step": 1205
1746
- },
1747
- {
1748
- "epoch": 0.82537517053206,
1749
- "grad_norm": 0.28732752799987793,
1750
- "learning_rate": 6.852404701203738e-05,
1751
- "loss": 0.017132116854190825,
1752
- "step": 1210
1753
- },
1754
- {
1755
- "epoch": 0.8287858117326057,
1756
- "grad_norm": 0.011728805489838123,
1757
- "learning_rate": 6.846634374162082e-05,
1758
- "loss": 0.043106210231781,
1759
- "step": 1215
1760
- },
1761
- {
1762
- "epoch": 0.8321964529331515,
1763
- "grad_norm": 0.06264204531908035,
1764
- "learning_rate": 6.84084090526529e-05,
1765
- "loss": 0.09258266687393188,
1766
- "step": 1220
1767
- },
1768
- {
1769
- "epoch": 0.8356070941336972,
1770
- "grad_norm": 0.024779995903372765,
1771
- "learning_rate": 6.835024337809278e-05,
1772
- "loss": 0.08440889716148377,
1773
- "step": 1225
1774
- },
1775
- {
1776
- "epoch": 0.8390177353342428,
1777
- "grad_norm": 0.3760814964771271,
1778
- "learning_rate": 6.829184715262579e-05,
1779
- "loss": 0.046157938241958615,
1780
- "step": 1230
1781
- },
1782
- {
1783
- "epoch": 0.8424283765347885,
1784
- "grad_norm": 0.41763243079185486,
1785
- "learning_rate": 6.823322081266027e-05,
1786
- "loss": 0.007576120644807815,
1787
- "step": 1235
1788
- },
1789
- {
1790
- "epoch": 0.8458390177353342,
1791
- "grad_norm": 0.20451563596725464,
1792
- "learning_rate": 6.817436479632423e-05,
1793
- "loss": 0.00685272142291069,
1794
- "step": 1240
1795
- },
1796
- {
1797
- "epoch": 0.8492496589358799,
1798
- "grad_norm": 0.19664883613586426,
1799
- "learning_rate": 6.811527954346208e-05,
1800
- "loss": 0.029371869564056397,
1801
- "step": 1245
1802
- },
1803
- {
1804
- "epoch": 0.8526603001364257,
1805
- "grad_norm": 0.18445859849452972,
1806
- "learning_rate": 6.805596549563143e-05,
1807
- "loss": 0.013169947266578674,
1808
- "step": 1250
1809
- },
1810
- {
1811
- "epoch": 0.8560709413369714,
1812
- "grad_norm": 0.25306227803230286,
1813
- "learning_rate": 6.799642309609968e-05,
1814
- "loss": 0.04840644598007202,
1815
- "step": 1255
1816
- },
1817
- {
1818
- "epoch": 0.859481582537517,
1819
- "grad_norm": 0.013680736534297466,
1820
- "learning_rate": 6.793665278984076e-05,
1821
- "loss": 0.005744677782058716,
1822
- "step": 1260
1823
- },
1824
- {
1825
- "epoch": 0.8628922237380627,
1826
- "grad_norm": 0.896000862121582,
1827
- "learning_rate": 6.78766550235318e-05,
1828
- "loss": 0.06524677276611328,
1829
- "step": 1265
1830
- },
1831
- {
1832
- "epoch": 0.8663028649386084,
1833
- "grad_norm": 0.028516558930277824,
1834
- "learning_rate": 6.781643024554982e-05,
1835
- "loss": 0.011343669146299362,
1836
- "step": 1270
1837
- },
1838
- {
1839
- "epoch": 0.8697135061391542,
1840
- "grad_norm": 0.8379983305931091,
1841
- "learning_rate": 6.775597890596829e-05,
1842
- "loss": 0.022122929990291595,
1843
- "step": 1275
1844
- },
1845
- {
1846
- "epoch": 0.8731241473396999,
1847
- "grad_norm": 1.4136202335357666,
1848
- "learning_rate": 6.769530145655389e-05,
1849
- "loss": 0.0679425597190857,
1850
- "step": 1280
1851
- },
1852
- {
1853
- "epoch": 0.8765347885402456,
1854
- "grad_norm": 1.3402701616287231,
1855
- "learning_rate": 6.763439835076303e-05,
1856
- "loss": 0.04459039568901062,
1857
- "step": 1285
1858
- },
1859
- {
1860
- "epoch": 0.8799454297407913,
1861
- "grad_norm": 1.0337740182876587,
1862
- "learning_rate": 6.757327004373852e-05,
1863
- "loss": 0.03138587772846222,
1864
- "step": 1290
1865
- },
1866
- {
1867
- "epoch": 0.883356070941337,
1868
- "grad_norm": 0.0886356458067894,
1869
- "learning_rate": 6.751191699230613e-05,
1870
- "loss": 0.008893608301877975,
1871
- "step": 1295
1872
- },
1873
- {
1874
- "epoch": 0.8867667121418826,
1875
- "grad_norm": 0.2752978801727295,
1876
- "learning_rate": 6.745033965497122e-05,
1877
- "loss": 0.036550650000572206,
1878
- "step": 1300
1879
- },
1880
- {
1881
- "epoch": 0.8901773533424284,
1882
- "grad_norm": 0.17057837545871735,
1883
- "learning_rate": 6.73885384919153e-05,
1884
- "loss": 0.34759960174560545,
1885
- "step": 1305
1886
- },
1887
- {
1888
- "epoch": 0.8935879945429741,
1889
- "grad_norm": 0.9223344326019287,
1890
- "learning_rate": 6.732651396499253e-05,
1891
- "loss": 0.017408999800682067,
1892
- "step": 1310
1893
- },
1894
- {
1895
- "epoch": 0.8969986357435198,
1896
- "grad_norm": 0.3093169033527374,
1897
- "learning_rate": 6.726426653772635e-05,
1898
- "loss": 0.05584460496902466,
1899
- "step": 1315
1900
- },
1901
- {
1902
- "epoch": 0.9004092769440655,
1903
- "grad_norm": 1.0157201290130615,
1904
- "learning_rate": 6.7201796675306e-05,
1905
- "loss": 0.023202185332775117,
1906
- "step": 1320
1907
- },
1908
- {
1909
- "epoch": 0.9038199181446112,
1910
- "grad_norm": 0.9780434370040894,
1911
- "learning_rate": 6.713910484458302e-05,
1912
- "loss": 0.029402348399162292,
1913
- "step": 1325
1914
- },
1915
- {
1916
- "epoch": 0.9072305593451568,
1917
- "grad_norm": 0.07956309616565704,
1918
- "learning_rate": 6.707619151406774e-05,
1919
- "loss": 0.02493150979280472,
1920
- "step": 1330
1921
- },
1922
- {
1923
- "epoch": 0.9106412005457026,
1924
- "grad_norm": 0.18366064131259918,
1925
- "learning_rate": 6.701305715392586e-05,
1926
- "loss": 0.06721556782722474,
1927
- "step": 1335
1928
- },
1929
- {
1930
- "epoch": 0.9140518417462483,
1931
- "grad_norm": 0.8265342116355896,
1932
- "learning_rate": 6.694970223597483e-05,
1933
- "loss": 0.03872359693050385,
1934
- "step": 1340
1935
- },
1936
- {
1937
- "epoch": 0.917462482946794,
1938
- "grad_norm": 1.9715158939361572,
1939
- "learning_rate": 6.688612723368042e-05,
1940
- "loss": 0.05604517459869385,
1941
- "step": 1345
1942
- },
1943
- {
1944
- "epoch": 0.9208731241473397,
1945
- "grad_norm": 0.06577733904123306,
1946
- "learning_rate": 6.682233262215312e-05,
1947
- "loss": 0.04941270649433136,
1948
- "step": 1350
1949
- },
1950
- {
1951
- "epoch": 0.9242837653478854,
1952
- "grad_norm": 0.2798021733760834,
1953
- "learning_rate": 6.67583188781446e-05,
1954
- "loss": 0.011715996265411376,
1955
- "step": 1355
1956
- },
1957
- {
1958
- "epoch": 0.927694406548431,
1959
- "grad_norm": 0.7599299550056458,
1960
- "learning_rate": 6.669408648004423e-05,
1961
- "loss": 0.030529171228408813,
1962
- "step": 1360
1963
- },
1964
- {
1965
- "epoch": 0.9311050477489768,
1966
- "grad_norm": 0.3893057405948639,
1967
- "learning_rate": 6.662963590787532e-05,
1968
- "loss": 0.06623916625976563,
1969
- "step": 1365
1970
- },
1971
- {
1972
- "epoch": 0.9345156889495225,
1973
- "grad_norm": 0.3796108365058899,
1974
- "learning_rate": 6.656496764329171e-05,
1975
- "loss": 0.02021588236093521,
1976
- "step": 1370
1977
- },
1978
- {
1979
- "epoch": 0.9379263301500682,
1980
- "grad_norm": 0.9708864688873291,
1981
- "learning_rate": 6.65000821695741e-05,
1982
- "loss": 0.05283964872360229,
1983
- "step": 1375
1984
- },
1985
- {
1986
- "epoch": 0.9413369713506139,
1987
- "grad_norm": 1.2553836107254028,
1988
- "learning_rate": 6.643497997162645e-05,
1989
- "loss": 0.11128103733062744,
1990
- "step": 1380
1991
- },
1992
- {
1993
- "epoch": 0.9447476125511596,
1994
- "grad_norm": 1.7390260696411133,
1995
- "learning_rate": 6.636966153597231e-05,
1996
- "loss": 0.051687347888946536,
1997
- "step": 1385
1998
- },
1999
- {
2000
- "epoch": 0.9481582537517054,
2001
- "grad_norm": 0.575423538684845,
2002
- "learning_rate": 6.630412735075128e-05,
2003
- "loss": 0.03732641041278839,
2004
- "step": 1390
2005
- },
2006
- {
2007
- "epoch": 0.951568894952251,
2008
- "grad_norm": 0.8504135608673096,
2009
- "learning_rate": 6.623837790571525e-05,
2010
- "loss": 0.038179832696914676,
2011
- "step": 1395
2012
- },
2013
- {
2014
- "epoch": 0.9549795361527967,
2015
- "grad_norm": 0.3777940273284912,
2016
- "learning_rate": 6.617241369222483e-05,
2017
- "loss": 0.01817839443683624,
2018
- "step": 1400
2019
- },
2020
- {
2021
- "epoch": 0.9583901773533424,
2022
- "grad_norm": 1.1219260692596436,
2023
- "learning_rate": 6.610623520324567e-05,
2024
- "loss": 0.0864151120185852,
2025
- "step": 1405
2026
- },
2027
- {
2028
- "epoch": 0.9618008185538881,
2029
- "grad_norm": 0.2320811152458191,
2030
- "learning_rate": 6.603984293334466e-05,
2031
- "loss": 0.0041168566793203356,
2032
- "step": 1410
2033
- },
2034
- {
2035
- "epoch": 0.9652114597544338,
2036
- "grad_norm": 0.5541130304336548,
2037
- "learning_rate": 6.597323737868642e-05,
2038
- "loss": 0.06572380065917968,
2039
- "step": 1415
2040
- },
2041
- {
2042
- "epoch": 0.9686221009549796,
2043
- "grad_norm": 0.1585462987422943,
2044
- "learning_rate": 6.590641903702944e-05,
2045
- "loss": 0.03849715292453766,
2046
- "step": 1420
2047
- },
2048
- {
2049
- "epoch": 0.9720327421555253,
2050
- "grad_norm": 1.7849252223968506,
2051
- "learning_rate": 6.583938840772245e-05,
2052
- "loss": 0.10304104089736939,
2053
- "step": 1425
2054
- },
2055
- {
2056
- "epoch": 0.975443383356071,
2057
- "grad_norm": 2.1307897567749023,
2058
- "learning_rate": 6.57721459917006e-05,
2059
- "loss": 0.036449754238128663,
2060
- "step": 1430
2061
- },
2062
- {
2063
- "epoch": 0.9788540245566166,
2064
- "grad_norm": 1.857226848602295,
2065
- "learning_rate": 6.570469229148184e-05,
2066
- "loss": 0.04441194534301758,
2067
- "step": 1435
2068
- },
2069
- {
2070
- "epoch": 0.9822646657571623,
2071
- "grad_norm": 0.27433109283447266,
2072
- "learning_rate": 6.563702781116302e-05,
2073
- "loss": 0.028930380940437317,
2074
- "step": 1440
2075
- },
2076
- {
2077
- "epoch": 0.985675306957708,
2078
- "grad_norm": 0.314373642206192,
2079
- "learning_rate": 6.556915305641629e-05,
2080
- "loss": 0.04347030222415924,
2081
- "step": 1445
2082
- },
2083
- {
2084
- "epoch": 0.9890859481582538,
2085
- "grad_norm": 0.3977321982383728,
2086
- "learning_rate": 6.550106853448513e-05,
2087
- "loss": 0.0386979341506958,
2088
- "step": 1450
2089
- },
2090
- {
2091
- "epoch": 0.9924965893587995,
2092
- "grad_norm": 1.6032801866531372,
2093
- "learning_rate": 6.543277475418074e-05,
2094
- "loss": 0.03199186325073242,
2095
- "step": 1455
2096
- },
2097
- {
2098
- "epoch": 0.9959072305593452,
2099
- "grad_norm": 1.1229087114334106,
2100
- "learning_rate": 6.53642722258781e-05,
2101
- "loss": 0.046002286672592166,
2102
- "step": 1460
2103
- },
2104
- {
2105
- "epoch": 0.9986357435197817,
2106
- "eval_loss": 0.05529617890715599,
2107
- "eval_runtime": 0.9152,
2108
- "eval_samples_per_second": 81.948,
2109
- "eval_steps_per_second": 2.185,
2110
- "step": 1464
2111
- },
2112
- {
2113
- "eval_cer_subset": 0.024662760594086387,
2114
- "eval_cer_subset_edit_distance": 181,
2115
- "eval_cer_subset_groups": 75,
2116
- "eval_cer_subset_items": 75,
2117
- "eval_cer_subset_ref_chars": 7339,
2118
- "step": 1464
2119
- }
2120
- ],
2121
- "logging_steps": 5,
2122
- "max_steps": 5864,
2123
- "num_input_tokens_seen": 0,
2124
- "num_train_epochs": 4,
2125
- "save_steps": 366,
2126
- "stateful_callbacks": {
2127
- "TrainerControl": {
2128
- "args": {
2129
- "should_epoch_stop": false,
2130
- "should_evaluate": false,
2131
- "should_log": false,
2132
- "should_save": true,
2133
- "should_training_stop": false
2134
- },
2135
- "attributes": {}
2136
- }
2137
- },
2138
- "total_flos": 1.5267574879617024e+16,
2139
- "train_batch_size": 2,
2140
- "trial_name": null,
2141
- "trial_params": null
2142
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1464/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bc6915d8d9dd9b5c9c17756c87ba7ec8221fd06789232d79225f9518167f0aa1
3
- size 5841
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/README.md DELETED
@@ -1,210 +0,0 @@
1
- ---
2
- base_model: unsloth/Meta-Llama-3.1-8B
3
- library_name: peft
4
- pipeline_tag: text-generation
5
- tags:
6
- - base_model:adapter:unsloth/Meta-Llama-3.1-8B
7
- - lora
8
- - sft
9
- - transformers
10
- - trl
11
- - unsloth
12
- ---
13
-
14
- # Model Card for Model ID
15
-
16
- <!-- Provide a quick summary of what the model is/does. -->
17
-
18
-
19
-
20
- ## Model Details
21
-
22
- ### Model Description
23
-
24
- <!-- Provide a longer summary of what this model is. -->
25
-
26
-
27
-
28
- - **Developed by:** [More Information Needed]
29
- - **Funded by [optional]:** [More Information Needed]
30
- - **Shared by [optional]:** [More Information Needed]
31
- - **Model type:** [More Information Needed]
32
- - **Language(s) (NLP):** [More Information Needed]
33
- - **License:** [More Information Needed]
34
- - **Finetuned from model [optional]:** [More Information Needed]
35
-
36
- ### Model Sources [optional]
37
-
38
- <!-- Provide the basic links for the model. -->
39
-
40
- - **Repository:** [More Information Needed]
41
- - **Paper [optional]:** [More Information Needed]
42
- - **Demo [optional]:** [More Information Needed]
43
-
44
- ## Uses
45
-
46
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
-
48
- ### Direct Use
49
-
50
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
-
52
- [More Information Needed]
53
-
54
- ### Downstream Use [optional]
55
-
56
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
-
58
- [More Information Needed]
59
-
60
- ### Out-of-Scope Use
61
-
62
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
-
64
- [More Information Needed]
65
-
66
- ## Bias, Risks, and Limitations
67
-
68
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
-
70
- [More Information Needed]
71
-
72
- ### Recommendations
73
-
74
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
-
76
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
-
78
- ## How to Get Started with the Model
79
-
80
- Use the code below to get started with the model.
81
-
82
- [More Information Needed]
83
-
84
- ## Training Details
85
-
86
- ### Training Data
87
-
88
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
-
90
- [More Information Needed]
91
-
92
- ### Training Procedure
93
-
94
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
-
96
- #### Preprocessing [optional]
97
-
98
- [More Information Needed]
99
-
100
-
101
- #### Training Hyperparameters
102
-
103
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
-
105
- #### Speeds, Sizes, Times [optional]
106
-
107
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
-
109
- [More Information Needed]
110
-
111
- ## Evaluation
112
-
113
- <!-- This section describes the evaluation protocols and provides the results. -->
114
-
115
- ### Testing Data, Factors & Metrics
116
-
117
- #### Testing Data
118
-
119
- <!-- This should link to a Dataset Card if possible. -->
120
-
121
- [More Information Needed]
122
-
123
- #### Factors
124
-
125
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
-
127
- [More Information Needed]
128
-
129
- #### Metrics
130
-
131
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
-
133
- [More Information Needed]
134
-
135
- ### Results
136
-
137
- [More Information Needed]
138
-
139
- #### Summary
140
-
141
-
142
-
143
- ## Model Examination [optional]
144
-
145
- <!-- Relevant interpretability work for the model goes here -->
146
-
147
- [More Information Needed]
148
-
149
- ## Environmental Impact
150
-
151
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
-
153
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
-
155
- - **Hardware Type:** [More Information Needed]
156
- - **Hours used:** [More Information Needed]
157
- - **Cloud Provider:** [More Information Needed]
158
- - **Compute Region:** [More Information Needed]
159
- - **Carbon Emitted:** [More Information Needed]
160
-
161
- ## Technical Specifications [optional]
162
-
163
- ### Model Architecture and Objective
164
-
165
- [More Information Needed]
166
-
167
- ### Compute Infrastructure
168
-
169
- [More Information Needed]
170
-
171
- #### Hardware
172
-
173
- [More Information Needed]
174
-
175
- #### Software
176
-
177
- [More Information Needed]
178
-
179
- ## Citation [optional]
180
-
181
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
-
183
- **BibTeX:**
184
-
185
- [More Information Needed]
186
-
187
- **APA:**
188
-
189
- [More Information Needed]
190
-
191
- ## Glossary [optional]
192
-
193
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
-
195
- [More Information Needed]
196
-
197
- ## More Information [optional]
198
-
199
- [More Information Needed]
200
-
201
- ## Model Card Authors [optional]
202
-
203
- [More Information Needed]
204
-
205
- ## Model Card Contact
206
-
207
- [More Information Needed]
208
- ### Framework versions
209
-
210
- - PEFT 0.18.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/adapter_config.json DELETED
@@ -1,50 +0,0 @@
1
- {
2
- "alora_invocation_tokens": null,
3
- "alpha_pattern": {},
4
- "arrow_config": null,
5
- "auto_mapping": {
6
- "base_model_class": "LlamaForCausalLM",
7
- "parent_library": "transformers.models.llama.modeling_llama",
8
- "unsloth_fixed": true
9
- },
10
- "base_model_name_or_path": "unsloth/Meta-Llama-3.1-8B",
11
- "bias": "none",
12
- "corda_config": null,
13
- "ensure_weight_tying": false,
14
- "eva_config": null,
15
- "exclude_modules": null,
16
- "fan_in_fan_out": false,
17
- "inference_mode": true,
18
- "init_lora_weights": true,
19
- "layer_replication": null,
20
- "layers_pattern": null,
21
- "layers_to_transform": null,
22
- "loftq_config": {},
23
- "lora_alpha": 16,
24
- "lora_bias": false,
25
- "lora_dropout": 0.0,
26
- "megatron_config": null,
27
- "megatron_core": "megatron.core",
28
- "modules_to_save": null,
29
- "peft_type": "LORA",
30
- "peft_version": "0.18.1",
31
- "qalora_group_size": 16,
32
- "r": 8,
33
- "rank_pattern": {},
34
- "revision": null,
35
- "target_modules": [
36
- "down_proj",
37
- "v_proj",
38
- "gate_proj",
39
- "up_proj",
40
- "o_proj",
41
- "q_proj",
42
- "k_proj"
43
- ],
44
- "target_parameters": null,
45
- "task_type": "CAUSAL_LM",
46
- "trainable_token_indices": null,
47
- "use_dora": false,
48
- "use_qalora": false,
49
- "use_rslora": false
50
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:eeae5e07a8d9b92052b6418ada0aeec6c226839a6e77f2fffb796f75c14516a9
3
- size 83945296
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/chat_template.jinja DELETED
@@ -1,12 +0,0 @@
1
- {{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ messages[0]['content'] + '
2
-
3
- ' }}{% set loop_messages = messages[1:] %}{% else %}{{ '' + '
4
-
5
- ' }}{% set loop_messages = messages %}{% endif %}{% for message in loop_messages %}{% if message['role'] == 'user' %}{{ '### Instruction:
6
- ' + message['content'] + '
7
-
8
- ' }}{% elif message['role'] == 'assistant' %}{{ '### Response:
9
- ' + message['content'] + eos_token + '
10
-
11
- ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '### Response:
12
- ' }}{% endif %}
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8b584a4e97475ef1d99387db902af3f22c6d714a86d5800599a9c5b4a696a072
3
- size 43127973
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:9ccb8eeb935749fc43744e0a5eeacdf6f0f10253be15266a497cbca0ffaa2573
3
- size 14645
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:adf5bf1180c4073f3c778c1ab49bbe225838d3c7885b46f176a1b30b405c7f03
3
- size 1465
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/tokenizer.json DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
3
- size 17209920
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/tokenizer_config.json DELETED
@@ -1,18 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "bos_token": "<|begin_of_text|>",
4
- "clean_up_tokenization_spaces": true,
5
- "eos_token": "<|end_of_text|>",
6
- "from_slow": true,
7
- "is_local": false,
8
- "legacy": false,
9
- "model_input_names": [
10
- "input_ids",
11
- "attention_mask"
12
- ],
13
- "model_max_length": 131072,
14
- "pad_token": "<|finetune_right_pad_id|>",
15
- "padding_side": "right",
16
- "tokenizer_class": "TokenizersBackend",
17
- "unk_token": null
18
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/trainer_state.json DELETED
@@ -1,2101 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 0.24995737425404946,
6
- "eval_steps": 1466,
7
- "global_step": 1466,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.0008525149190110827,
14
- "grad_norm": 1.6653118133544922,
15
- "learning_rate": 6.382978723404255e-07,
16
- "loss": 0.6878558158874511,
17
- "step": 5
18
- },
19
- {
20
- "epoch": 0.0017050298380221654,
21
- "grad_norm": 1.6072094440460205,
22
- "learning_rate": 1.4361702127659573e-06,
23
- "loss": 0.7222867965698242,
24
- "step": 10
25
- },
26
- {
27
- "epoch": 0.0025575447570332483,
28
- "grad_norm": 1.8327608108520508,
29
- "learning_rate": 2.2340425531914894e-06,
30
- "loss": 0.6977188110351562,
31
- "step": 15
32
- },
33
- {
34
- "epoch": 0.0034100596760443308,
35
- "grad_norm": 1.7263766527175903,
36
- "learning_rate": 3.0319148936170214e-06,
37
- "loss": 0.6928215026855469,
38
- "step": 20
39
- },
40
- {
41
- "epoch": 0.004262574595055414,
42
- "grad_norm": 1.8482989072799683,
43
- "learning_rate": 3.829787234042553e-06,
44
- "loss": 0.6887872695922852,
45
- "step": 25
46
- },
47
- {
48
- "epoch": 0.005115089514066497,
49
- "grad_norm": 2.0272440910339355,
50
- "learning_rate": 4.627659574468085e-06,
51
- "loss": 0.6714544296264648,
52
- "step": 30
53
- },
54
- {
55
- "epoch": 0.005967604433077579,
56
- "grad_norm": 2.4286959171295166,
57
- "learning_rate": 5.425531914893616e-06,
58
- "loss": 0.6025971412658692,
59
- "step": 35
60
- },
61
- {
62
- "epoch": 0.0068201193520886615,
63
- "grad_norm": 2.26863694190979,
64
- "learning_rate": 6.223404255319148e-06,
65
- "loss": 0.5200423240661621,
66
- "step": 40
67
- },
68
- {
69
- "epoch": 0.0076726342710997444,
70
- "grad_norm": 2.0572476387023926,
71
- "learning_rate": 7.02127659574468e-06,
72
- "loss": 0.3741515874862671,
73
- "step": 45
74
- },
75
- {
76
- "epoch": 0.008525149190110827,
77
- "grad_norm": 0.9048103094100952,
78
- "learning_rate": 7.819148936170211e-06,
79
- "loss": 0.25923154354095457,
80
- "step": 50
81
- },
82
- {
83
- "epoch": 0.00937766410912191,
84
- "grad_norm": 0.5942133069038391,
85
- "learning_rate": 8.617021276595744e-06,
86
- "loss": 0.18589640855789186,
87
- "step": 55
88
- },
89
- {
90
- "epoch": 0.010230179028132993,
91
- "grad_norm": 0.40469691157341003,
92
- "learning_rate": 9.414893617021275e-06,
93
- "loss": 0.14508966207504273,
94
- "step": 60
95
- },
96
- {
97
- "epoch": 0.011082693947144074,
98
- "grad_norm": 0.319180428981781,
99
- "learning_rate": 1.0212765957446808e-05,
100
- "loss": 0.1287898302078247,
101
- "step": 65
102
- },
103
- {
104
- "epoch": 0.011935208866155157,
105
- "grad_norm": 0.2620982229709625,
106
- "learning_rate": 1.101063829787234e-05,
107
- "loss": 0.10825083255767823,
108
- "step": 70
109
- },
110
- {
111
- "epoch": 0.01278772378516624,
112
- "grad_norm": 0.2702063024044037,
113
- "learning_rate": 1.1808510638297872e-05,
114
- "loss": 0.09140915870666504,
115
- "step": 75
116
- },
117
- {
118
- "epoch": 0.013640238704177323,
119
- "grad_norm": 0.21019567549228668,
120
- "learning_rate": 1.2606382978723403e-05,
121
- "loss": 0.07847741842269898,
122
- "step": 80
123
- },
124
- {
125
- "epoch": 0.014492753623188406,
126
- "grad_norm": 0.27865487337112427,
127
- "learning_rate": 1.3404255319148936e-05,
128
- "loss": 0.0681579053401947,
129
- "step": 85
130
- },
131
- {
132
- "epoch": 0.015345268542199489,
133
- "grad_norm": 0.2561706006526947,
134
- "learning_rate": 1.4202127659574466e-05,
135
- "loss": 0.06208043098449707,
136
- "step": 90
137
- },
138
- {
139
- "epoch": 0.01619778346121057,
140
- "grad_norm": 0.33415308594703674,
141
- "learning_rate": 1.4999999999999999e-05,
142
- "loss": 0.05436077117919922,
143
- "step": 95
144
- },
145
- {
146
- "epoch": 0.017050298380221655,
147
- "grad_norm": 0.20705723762512207,
148
- "learning_rate": 1.579787234042553e-05,
149
- "loss": 0.061427068710327146,
150
- "step": 100
151
- },
152
- {
153
- "epoch": 0.017902813299232736,
154
- "grad_norm": 0.26269420981407166,
155
- "learning_rate": 1.659574468085106e-05,
156
- "loss": 0.04944279193878174,
157
- "step": 105
158
- },
159
- {
160
- "epoch": 0.01875532821824382,
161
- "grad_norm": 0.256533682346344,
162
- "learning_rate": 1.7393617021276596e-05,
163
- "loss": 0.0463131994009018,
164
- "step": 110
165
- },
166
- {
167
- "epoch": 0.0196078431372549,
168
- "grad_norm": 0.25077348947525024,
169
- "learning_rate": 1.8191489361702127e-05,
170
- "loss": 0.036723154783248904,
171
- "step": 115
172
- },
173
- {
174
- "epoch": 0.020460358056265986,
175
- "grad_norm": 0.2801378071308136,
176
- "learning_rate": 1.8989361702127655e-05,
177
- "loss": 0.04308137893676758,
178
- "step": 120
179
- },
180
- {
181
- "epoch": 0.021312872975277068,
182
- "grad_norm": 0.23140908777713776,
183
- "learning_rate": 1.978723404255319e-05,
184
- "loss": 0.0391487717628479,
185
- "step": 125
186
- },
187
- {
188
- "epoch": 0.02216538789428815,
189
- "grad_norm": 0.4250975251197815,
190
- "learning_rate": 2.0585106382978724e-05,
191
- "loss": 0.03745899498462677,
192
- "step": 130
193
- },
194
- {
195
- "epoch": 0.023017902813299233,
196
- "grad_norm": 0.24533668160438538,
197
- "learning_rate": 2.1382978723404255e-05,
198
- "loss": 0.03649275898933411,
199
- "step": 135
200
- },
201
- {
202
- "epoch": 0.023870417732310314,
203
- "grad_norm": 0.2309182733297348,
204
- "learning_rate": 2.2180851063829783e-05,
205
- "loss": 0.036090463399887085,
206
- "step": 140
207
- },
208
- {
209
- "epoch": 0.0247229326513214,
210
- "grad_norm": 0.28087928891181946,
211
- "learning_rate": 2.2978723404255317e-05,
212
- "loss": 0.03179733753204346,
213
- "step": 145
214
- },
215
- {
216
- "epoch": 0.02557544757033248,
217
- "grad_norm": 0.2785134017467499,
218
- "learning_rate": 2.377659574468085e-05,
219
- "loss": 0.02794267237186432,
220
- "step": 150
221
- },
222
- {
223
- "epoch": 0.026427962489343565,
224
- "grad_norm": 0.20468176901340485,
225
- "learning_rate": 2.457446808510638e-05,
226
- "loss": 0.025854837894439698,
227
- "step": 155
228
- },
229
- {
230
- "epoch": 0.027280477408354646,
231
- "grad_norm": 0.4069538116455078,
232
- "learning_rate": 2.537234042553191e-05,
233
- "loss": 0.03182780146598816,
234
- "step": 160
235
- },
236
- {
237
- "epoch": 0.028132992327365727,
238
- "grad_norm": 0.3025060296058655,
239
- "learning_rate": 2.6170212765957446e-05,
240
- "loss": 0.027975103259086607,
241
- "step": 165
242
- },
243
- {
244
- "epoch": 0.028985507246376812,
245
- "grad_norm": 0.22387781739234924,
246
- "learning_rate": 2.6968085106382977e-05,
247
- "loss": 0.025766396522521974,
248
- "step": 170
249
- },
250
- {
251
- "epoch": 0.029838022165387893,
252
- "grad_norm": 0.15060213208198547,
253
- "learning_rate": 2.7765957446808508e-05,
254
- "loss": 0.025661858916282653,
255
- "step": 175
256
- },
257
- {
258
- "epoch": 0.030690537084398978,
259
- "grad_norm": 0.2912445366382599,
260
- "learning_rate": 2.856382978723404e-05,
261
- "loss": 0.02397846579551697,
262
- "step": 180
263
- },
264
- {
265
- "epoch": 0.03154305200341006,
266
- "grad_norm": 0.19390830397605896,
267
- "learning_rate": 2.9361702127659574e-05,
268
- "loss": 0.023744897544384004,
269
- "step": 185
270
- },
271
- {
272
- "epoch": 0.03239556692242114,
273
- "grad_norm": 0.24036464095115662,
274
- "learning_rate": 3.0159574468085105e-05,
275
- "loss": 0.0238590270280838,
276
- "step": 190
277
- },
278
- {
279
- "epoch": 0.03324808184143223,
280
- "grad_norm": 0.2793026566505432,
281
- "learning_rate": 3.0957446808510636e-05,
282
- "loss": 0.025464403629302978,
283
- "step": 195
284
- },
285
- {
286
- "epoch": 0.03410059676044331,
287
- "grad_norm": 0.21164429187774658,
288
- "learning_rate": 3.175531914893617e-05,
289
- "loss": 0.020692554116249085,
290
- "step": 200
291
- },
292
- {
293
- "epoch": 0.03495311167945439,
294
- "grad_norm": 0.3742266297340393,
295
- "learning_rate": 3.25531914893617e-05,
296
- "loss": 0.02315486818552017,
297
- "step": 205
298
- },
299
- {
300
- "epoch": 0.03580562659846547,
301
- "grad_norm": 0.2690248191356659,
302
- "learning_rate": 3.3351063829787226e-05,
303
- "loss": 0.0174571692943573,
304
- "step": 210
305
- },
306
- {
307
- "epoch": 0.03665814151747655,
308
- "grad_norm": 0.3727855980396271,
309
- "learning_rate": 3.414893617021276e-05,
310
- "loss": 0.021705827116966246,
311
- "step": 215
312
- },
313
- {
314
- "epoch": 0.03751065643648764,
315
- "grad_norm": 0.27461397647857666,
316
- "learning_rate": 3.4946808510638296e-05,
317
- "loss": 0.028134092688560486,
318
- "step": 220
319
- },
320
- {
321
- "epoch": 0.03836317135549872,
322
- "grad_norm": 0.1723722219467163,
323
- "learning_rate": 3.574468085106383e-05,
324
- "loss": 0.02093944847583771,
325
- "step": 225
326
- },
327
- {
328
- "epoch": 0.0392156862745098,
329
- "grad_norm": 0.17161321640014648,
330
- "learning_rate": 3.654255319148936e-05,
331
- "loss": 0.021984823048114777,
332
- "step": 230
333
- },
334
- {
335
- "epoch": 0.040068201193520885,
336
- "grad_norm": 0.203886479139328,
337
- "learning_rate": 3.734042553191489e-05,
338
- "loss": 0.02234097272157669,
339
- "step": 235
340
- },
341
- {
342
- "epoch": 0.04092071611253197,
343
- "grad_norm": 0.2021923065185547,
344
- "learning_rate": 3.813829787234042e-05,
345
- "loss": 0.02671273946762085,
346
- "step": 240
347
- },
348
- {
349
- "epoch": 0.041773231031543054,
350
- "grad_norm": 0.24510027468204498,
351
- "learning_rate": 3.8936170212765955e-05,
352
- "loss": 0.016925157606601716,
353
- "step": 245
354
- },
355
- {
356
- "epoch": 0.042625745950554135,
357
- "grad_norm": 0.191350057721138,
358
- "learning_rate": 3.973404255319149e-05,
359
- "loss": 0.02067229151725769,
360
- "step": 250
361
- },
362
- {
363
- "epoch": 0.043478260869565216,
364
- "grad_norm": 0.20248189568519592,
365
- "learning_rate": 4.053191489361702e-05,
366
- "loss": 0.01805101931095123,
367
- "step": 255
368
- },
369
- {
370
- "epoch": 0.0443307757885763,
371
- "grad_norm": 0.22623269259929657,
372
- "learning_rate": 4.1329787234042545e-05,
373
- "loss": 0.019811727106571198,
374
- "step": 260
375
- },
376
- {
377
- "epoch": 0.045183290707587385,
378
- "grad_norm": 0.2050849199295044,
379
- "learning_rate": 4.2127659574468086e-05,
380
- "loss": 0.017767781019210817,
381
- "step": 265
382
- },
383
- {
384
- "epoch": 0.04603580562659847,
385
- "grad_norm": 0.24867363274097443,
386
- "learning_rate": 4.2925531914893614e-05,
387
- "loss": 0.02127009928226471,
388
- "step": 270
389
- },
390
- {
391
- "epoch": 0.04688832054560955,
392
- "grad_norm": 0.20721694827079773,
393
- "learning_rate": 4.372340425531914e-05,
394
- "loss": 0.022770658135414124,
395
- "step": 275
396
- },
397
- {
398
- "epoch": 0.04774083546462063,
399
- "grad_norm": 0.19511370360851288,
400
- "learning_rate": 4.452127659574468e-05,
401
- "loss": 0.02165296971797943,
402
- "step": 280
403
- },
404
- {
405
- "epoch": 0.04859335038363171,
406
- "grad_norm": 0.19066740572452545,
407
- "learning_rate": 4.5319148936170204e-05,
408
- "loss": 0.020857495069503785,
409
- "step": 285
410
- },
411
- {
412
- "epoch": 0.0494458653026428,
413
- "grad_norm": 0.14217901229858398,
414
- "learning_rate": 4.6117021276595746e-05,
415
- "loss": 0.016413891315460206,
416
- "step": 290
417
- },
418
- {
419
- "epoch": 0.05029838022165388,
420
- "grad_norm": 0.24177870154380798,
421
- "learning_rate": 4.6914893617021274e-05,
422
- "loss": 0.01902117133140564,
423
- "step": 295
424
- },
425
- {
426
- "epoch": 0.05115089514066496,
427
- "grad_norm": 0.2742858827114105,
428
- "learning_rate": 4.77127659574468e-05,
429
- "loss": 0.016554126143455507,
430
- "step": 300
431
- },
432
- {
433
- "epoch": 0.05200341005967604,
434
- "grad_norm": 0.24519385397434235,
435
- "learning_rate": 4.851063829787234e-05,
436
- "loss": 0.022036357223987578,
437
- "step": 305
438
- },
439
- {
440
- "epoch": 0.05285592497868713,
441
- "grad_norm": 0.296572208404541,
442
- "learning_rate": 4.930851063829787e-05,
443
- "loss": 0.0196581095457077,
444
- "step": 310
445
- },
446
- {
447
- "epoch": 0.05370843989769821,
448
- "grad_norm": 0.17092008888721466,
449
- "learning_rate": 5.01063829787234e-05,
450
- "loss": 0.015453743934631347,
451
- "step": 315
452
- },
453
- {
454
- "epoch": 0.05456095481670929,
455
- "grad_norm": 0.17074067890644073,
456
- "learning_rate": 5.090425531914893e-05,
457
- "loss": 0.013983796536922454,
458
- "step": 320
459
- },
460
- {
461
- "epoch": 0.05541346973572037,
462
- "grad_norm": 0.27302470803260803,
463
- "learning_rate": 5.170212765957446e-05,
464
- "loss": 0.015387402474880218,
465
- "step": 325
466
- },
467
- {
468
- "epoch": 0.056265984654731455,
469
- "grad_norm": 0.2603592872619629,
470
- "learning_rate": 5.2499999999999995e-05,
471
- "loss": 0.013557825982570649,
472
- "step": 330
473
- },
474
- {
475
- "epoch": 0.05711849957374254,
476
- "grad_norm": 0.23079948127269745,
477
- "learning_rate": 5.329787234042553e-05,
478
- "loss": 0.014809322357177735,
479
- "step": 335
480
- },
481
- {
482
- "epoch": 0.057971014492753624,
483
- "grad_norm": 0.15029627084732056,
484
- "learning_rate": 5.409574468085106e-05,
485
- "loss": 0.018204084038734435,
486
- "step": 340
487
- },
488
- {
489
- "epoch": 0.058823529411764705,
490
- "grad_norm": 0.2930934429168701,
491
- "learning_rate": 5.4893617021276586e-05,
492
- "loss": 0.02035558819770813,
493
- "step": 345
494
- },
495
- {
496
- "epoch": 0.059676044330775786,
497
- "grad_norm": 0.19710905849933624,
498
- "learning_rate": 5.569148936170213e-05,
499
- "loss": 0.015025021135807037,
500
- "step": 350
501
- },
502
- {
503
- "epoch": 0.060528559249786874,
504
- "grad_norm": 0.13316969573497772,
505
- "learning_rate": 5.6489361702127655e-05,
506
- "loss": 0.01258893460035324,
507
- "step": 355
508
- },
509
- {
510
- "epoch": 0.061381074168797956,
511
- "grad_norm": 0.30591729283332825,
512
- "learning_rate": 5.728723404255319e-05,
513
- "loss": 0.0187319278717041,
514
- "step": 360
515
- },
516
- {
517
- "epoch": 0.06223358908780904,
518
- "grad_norm": 0.22696663439273834,
519
- "learning_rate": 5.808510638297872e-05,
520
- "loss": 0.015805599093437196,
521
- "step": 365
522
- },
523
- {
524
- "epoch": 0.06308610400682012,
525
- "grad_norm": 0.2800130546092987,
526
- "learning_rate": 5.8882978723404245e-05,
527
- "loss": 0.01719983071088791,
528
- "step": 370
529
- },
530
- {
531
- "epoch": 0.0639386189258312,
532
- "grad_norm": 0.20671235024929047,
533
- "learning_rate": 5.9680851063829786e-05,
534
- "loss": 0.014449423551559449,
535
- "step": 375
536
- },
537
- {
538
- "epoch": 0.06479113384484228,
539
- "grad_norm": 0.1630883365869522,
540
- "learning_rate": 6.0478723404255314e-05,
541
- "loss": 0.017153808474540712,
542
- "step": 380
543
- },
544
- {
545
- "epoch": 0.06564364876385337,
546
- "grad_norm": 0.18699344992637634,
547
- "learning_rate": 6.127659574468084e-05,
548
- "loss": 0.015558701753616334,
549
- "step": 385
550
- },
551
- {
552
- "epoch": 0.06649616368286446,
553
- "grad_norm": 0.15257929265499115,
554
- "learning_rate": 6.207446808510638e-05,
555
- "loss": 0.018240168690681458,
556
- "step": 390
557
- },
558
- {
559
- "epoch": 0.06734867860187553,
560
- "grad_norm": 0.19658304750919342,
561
- "learning_rate": 6.287234042553191e-05,
562
- "loss": 0.015907022356987,
563
- "step": 395
564
- },
565
- {
566
- "epoch": 0.06820119352088662,
567
- "grad_norm": 0.3743384778499603,
568
- "learning_rate": 6.367021276595743e-05,
569
- "loss": 0.014921861886978149,
570
- "step": 400
571
- },
572
- {
573
- "epoch": 0.06905370843989769,
574
- "grad_norm": 0.254724383354187,
575
- "learning_rate": 6.446808510638298e-05,
576
- "loss": 0.018739913403987885,
577
- "step": 405
578
- },
579
- {
580
- "epoch": 0.06990622335890878,
581
- "grad_norm": 0.19365151226520538,
582
- "learning_rate": 6.52659574468085e-05,
583
- "loss": 0.018984711170196532,
584
- "step": 410
585
- },
586
- {
587
- "epoch": 0.07075873827791987,
588
- "grad_norm": 0.20728597044944763,
589
- "learning_rate": 6.606382978723404e-05,
590
- "loss": 0.01524859368801117,
591
- "step": 415
592
- },
593
- {
594
- "epoch": 0.07161125319693094,
595
- "grad_norm": 0.1415344476699829,
596
- "learning_rate": 6.686170212765957e-05,
597
- "loss": 0.014004582166671753,
598
- "step": 420
599
- },
600
- {
601
- "epoch": 0.07246376811594203,
602
- "grad_norm": 0.14169375598430634,
603
- "learning_rate": 6.765957446808509e-05,
604
- "loss": 0.013503608107566834,
605
- "step": 425
606
- },
607
- {
608
- "epoch": 0.0733162830349531,
609
- "grad_norm": 0.22518369555473328,
610
- "learning_rate": 6.845744680851064e-05,
611
- "loss": 0.017587104439735414,
612
- "step": 430
613
- },
614
- {
615
- "epoch": 0.0741687979539642,
616
- "grad_norm": 0.1091267541050911,
617
- "learning_rate": 6.925531914893616e-05,
618
- "loss": 0.013890406489372254,
619
- "step": 435
620
- },
621
- {
622
- "epoch": 0.07502131287297528,
623
- "grad_norm": 0.24982722103595734,
624
- "learning_rate": 7.00531914893617e-05,
625
- "loss": 0.01599075198173523,
626
- "step": 440
627
- },
628
- {
629
- "epoch": 0.07587382779198636,
630
- "grad_norm": 0.22311007976531982,
631
- "learning_rate": 7.085106382978723e-05,
632
- "loss": 0.014870230853557587,
633
- "step": 445
634
- },
635
- {
636
- "epoch": 0.07672634271099744,
637
- "grad_norm": 0.27064985036849976,
638
- "learning_rate": 7.164893617021276e-05,
639
- "loss": 0.019213163852691652,
640
- "step": 450
641
- },
642
- {
643
- "epoch": 0.07757885763000852,
644
- "grad_norm": 0.16876882314682007,
645
- "learning_rate": 7.244680851063829e-05,
646
- "loss": 0.014935044944286347,
647
- "step": 455
648
- },
649
- {
650
- "epoch": 0.0784313725490196,
651
- "grad_norm": 0.18644963204860687,
652
- "learning_rate": 7.324468085106382e-05,
653
- "loss": 0.013823372125625611,
654
- "step": 460
655
- },
656
- {
657
- "epoch": 0.0792838874680307,
658
- "grad_norm": 0.13067972660064697,
659
- "learning_rate": 7.404255319148935e-05,
660
- "loss": 0.013839980959892273,
661
- "step": 465
662
- },
663
- {
664
- "epoch": 0.08013640238704177,
665
- "grad_norm": 0.0976850613951683,
666
- "learning_rate": 7.484042553191489e-05,
667
- "loss": 0.012782379984855652,
668
- "step": 470
669
- },
670
- {
671
- "epoch": 0.08098891730605286,
672
- "grad_norm": 0.2523496150970459,
673
- "learning_rate": 7.499999439800074e-05,
674
- "loss": 0.013075053691864014,
675
- "step": 475
676
- },
677
- {
678
- "epoch": 0.08184143222506395,
679
- "grad_norm": 0.18349343538284302,
680
- "learning_rate": 7.499997163988164e-05,
681
- "loss": 0.01753528565168381,
682
- "step": 480
683
- },
684
- {
685
- "epoch": 0.08269394714407502,
686
- "grad_norm": 0.16528834402561188,
687
- "learning_rate": 7.499993137552834e-05,
688
- "loss": 0.012987489998340606,
689
- "step": 485
690
- },
691
- {
692
- "epoch": 0.08354646206308611,
693
- "grad_norm": 0.1918938159942627,
694
- "learning_rate": 7.499987360495964e-05,
695
- "loss": 0.018496687710285186,
696
- "step": 490
697
- },
698
- {
699
- "epoch": 0.08439897698209718,
700
- "grad_norm": 0.11452502012252808,
701
- "learning_rate": 7.499979832820255e-05,
702
- "loss": 0.015578755736351013,
703
- "step": 495
704
- },
705
- {
706
- "epoch": 0.08525149190110827,
707
- "grad_norm": 0.2084985226392746,
708
- "learning_rate": 7.499970554529216e-05,
709
- "loss": 0.014264023303985596,
710
- "step": 500
711
- },
712
- {
713
- "epoch": 0.08610400682011936,
714
- "grad_norm": 0.13777600228786469,
715
- "learning_rate": 7.49995952562718e-05,
716
- "loss": 0.014527246356010437,
717
- "step": 505
718
- },
719
- {
720
- "epoch": 0.08695652173913043,
721
- "grad_norm": 0.2515336275100708,
722
- "learning_rate": 7.499946746119296e-05,
723
- "loss": 0.019042417407035828,
724
- "step": 510
725
- },
726
- {
727
- "epoch": 0.08780903665814152,
728
- "grad_norm": 0.12859591841697693,
729
- "learning_rate": 7.499932216011531e-05,
730
- "loss": 0.01340993344783783,
731
- "step": 515
732
- },
733
- {
734
- "epoch": 0.0886615515771526,
735
- "grad_norm": 0.15603038668632507,
736
- "learning_rate": 7.499915935310667e-05,
737
- "loss": 0.01645509898662567,
738
- "step": 520
739
- },
740
- {
741
- "epoch": 0.08951406649616368,
742
- "grad_norm": 0.1493784785270691,
743
- "learning_rate": 7.499897904024303e-05,
744
- "loss": 0.016503919661045075,
745
- "step": 525
746
- },
747
- {
748
- "epoch": 0.09036658141517477,
749
- "grad_norm": 0.14551150798797607,
750
- "learning_rate": 7.499878122160858e-05,
751
- "loss": 0.013891169428825378,
752
- "step": 530
753
- },
754
- {
755
- "epoch": 0.09121909633418585,
756
- "grad_norm": 0.12197990715503693,
757
- "learning_rate": 7.499856589729569e-05,
758
- "loss": 0.01531532108783722,
759
- "step": 535
760
- },
761
- {
762
- "epoch": 0.09207161125319693,
763
- "grad_norm": 0.24787496030330658,
764
- "learning_rate": 7.499833306740485e-05,
765
- "loss": 0.016374409198760986,
766
- "step": 540
767
- },
768
- {
769
- "epoch": 0.09292412617220801,
770
- "grad_norm": 0.10902808606624603,
771
- "learning_rate": 7.499808273204476e-05,
772
- "loss": 0.013505718111991883,
773
- "step": 545
774
- },
775
- {
776
- "epoch": 0.0937766410912191,
777
- "grad_norm": 0.16540755331516266,
778
- "learning_rate": 7.499781489133228e-05,
779
- "loss": 0.016257329285144805,
780
- "step": 550
781
- },
782
- {
783
- "epoch": 0.09462915601023018,
784
- "grad_norm": 0.1228717565536499,
785
- "learning_rate": 7.499752954539245e-05,
786
- "loss": 0.011345921456813813,
787
- "step": 555
788
- },
789
- {
790
- "epoch": 0.09548167092924126,
791
- "grad_norm": 0.12704797089099884,
792
- "learning_rate": 7.49972266943585e-05,
793
- "loss": 0.014449910819530487,
794
- "step": 560
795
- },
796
- {
797
- "epoch": 0.09633418584825235,
798
- "grad_norm": 0.2014201581478119,
799
- "learning_rate": 7.499690633837178e-05,
800
- "loss": 0.01456935852766037,
801
- "step": 565
802
- },
803
- {
804
- "epoch": 0.09718670076726342,
805
- "grad_norm": 0.17480657994747162,
806
- "learning_rate": 7.499656847758187e-05,
807
- "loss": 0.014413098990917205,
808
- "step": 570
809
- },
810
- {
811
- "epoch": 0.09803921568627451,
812
- "grad_norm": 0.18946893513202667,
813
- "learning_rate": 7.499621311214646e-05,
814
- "loss": 0.0139508917927742,
815
- "step": 575
816
- },
817
- {
818
- "epoch": 0.0988917306052856,
819
- "grad_norm": 0.15367820858955383,
820
- "learning_rate": 7.499584024223149e-05,
821
- "loss": 0.014091435074806213,
822
- "step": 580
823
- },
824
- {
825
- "epoch": 0.09974424552429667,
826
- "grad_norm": 0.15018688142299652,
827
- "learning_rate": 7.499544986801099e-05,
828
- "loss": 0.016737687587738036,
829
- "step": 585
830
- },
831
- {
832
- "epoch": 0.10059676044330776,
833
- "grad_norm": 0.17522579431533813,
834
- "learning_rate": 7.499504198966722e-05,
835
- "loss": 0.015822765231132508,
836
- "step": 590
837
- },
838
- {
839
- "epoch": 0.10144927536231885,
840
- "grad_norm": 0.12634818255901337,
841
- "learning_rate": 7.499461660739059e-05,
842
- "loss": 0.015363042056560517,
843
- "step": 595
844
- },
845
- {
846
- "epoch": 0.10230179028132992,
847
- "grad_norm": 0.12466495484113693,
848
- "learning_rate": 7.499417372137968e-05,
849
- "loss": 0.013208043575286866,
850
- "step": 600
851
- },
852
- {
853
- "epoch": 0.10315430520034101,
854
- "grad_norm": 0.18877951800823212,
855
- "learning_rate": 7.499371333184125e-05,
856
- "loss": 0.016261917352676392,
857
- "step": 605
858
- },
859
- {
860
- "epoch": 0.10400682011935208,
861
- "grad_norm": 0.21521399915218353,
862
- "learning_rate": 7.49932354389902e-05,
863
- "loss": 0.013861049711704255,
864
- "step": 610
865
- },
866
- {
867
- "epoch": 0.10485933503836317,
868
- "grad_norm": 0.1375613659620285,
869
- "learning_rate": 7.499274004304964e-05,
870
- "loss": 0.015597744286060334,
871
- "step": 615
872
- },
873
- {
874
- "epoch": 0.10571184995737426,
875
- "grad_norm": 0.16078200936317444,
876
- "learning_rate": 7.499222714425087e-05,
877
- "loss": 0.018308353424072266,
878
- "step": 620
879
- },
880
- {
881
- "epoch": 0.10656436487638533,
882
- "grad_norm": 0.15485632419586182,
883
- "learning_rate": 7.499169674283328e-05,
884
- "loss": 0.015836401283740996,
885
- "step": 625
886
- },
887
- {
888
- "epoch": 0.10741687979539642,
889
- "grad_norm": 0.14019452035427094,
890
- "learning_rate": 7.499114883904451e-05,
891
- "loss": 0.014591602981090546,
892
- "step": 630
893
- },
894
- {
895
- "epoch": 0.1082693947144075,
896
- "grad_norm": 0.2042212039232254,
897
- "learning_rate": 7.499058343314031e-05,
898
- "loss": 0.01718664914369583,
899
- "step": 635
900
- },
901
- {
902
- "epoch": 0.10912190963341858,
903
- "grad_norm": 0.13801656663417816,
904
- "learning_rate": 7.499000052538467e-05,
905
- "loss": 0.015579727292060853,
906
- "step": 640
907
- },
908
- {
909
- "epoch": 0.10997442455242967,
910
- "grad_norm": 0.12787523865699768,
911
- "learning_rate": 7.498940011604968e-05,
912
- "loss": 0.012525486946105956,
913
- "step": 645
914
- },
915
- {
916
- "epoch": 0.11082693947144075,
917
- "grad_norm": 0.1752539724111557,
918
- "learning_rate": 7.498878220541564e-05,
919
- "loss": 0.014286568760871888,
920
- "step": 650
921
- },
922
- {
923
- "epoch": 0.11167945439045183,
924
- "grad_norm": 0.12485212087631226,
925
- "learning_rate": 7.498814679377101e-05,
926
- "loss": 0.01482405811548233,
927
- "step": 655
928
- },
929
- {
930
- "epoch": 0.11253196930946291,
931
- "grad_norm": 0.10980220139026642,
932
- "learning_rate": 7.498749388141243e-05,
933
- "loss": 0.01597980558872223,
934
- "step": 660
935
- },
936
- {
937
- "epoch": 0.113384484228474,
938
- "grad_norm": 0.09144977480173111,
939
- "learning_rate": 7.498682346864469e-05,
940
- "loss": 0.011583130061626434,
941
- "step": 665
942
- },
943
- {
944
- "epoch": 0.11423699914748509,
945
- "grad_norm": 0.11955336481332779,
946
- "learning_rate": 7.498613555578076e-05,
947
- "loss": 0.013009116053581238,
948
- "step": 670
949
- },
950
- {
951
- "epoch": 0.11508951406649616,
952
- "grad_norm": 0.14505070447921753,
953
- "learning_rate": 7.49854301431418e-05,
954
- "loss": 0.013878281414508819,
955
- "step": 675
956
- },
957
- {
958
- "epoch": 0.11594202898550725,
959
- "grad_norm": 0.15674598515033722,
960
- "learning_rate": 7.49847072310571e-05,
961
- "loss": 0.014717698097229004,
962
- "step": 680
963
- },
964
- {
965
- "epoch": 0.11679454390451834,
966
- "grad_norm": 0.16684003174304962,
967
- "learning_rate": 7.498396681986413e-05,
968
- "loss": 0.015567027032375336,
969
- "step": 685
970
- },
971
- {
972
- "epoch": 0.11764705882352941,
973
- "grad_norm": 0.13314439356327057,
974
- "learning_rate": 7.498320890990857e-05,
975
- "loss": 0.017679129540920258,
976
- "step": 690
977
- },
978
- {
979
- "epoch": 0.1184995737425405,
980
- "grad_norm": 0.1099700927734375,
981
- "learning_rate": 7.498243350154423e-05,
982
- "loss": 0.015660777688026428,
983
- "step": 695
984
- },
985
- {
986
- "epoch": 0.11935208866155157,
987
- "grad_norm": 0.09305288642644882,
988
- "learning_rate": 7.498164059513307e-05,
989
- "loss": 0.011864218115806579,
990
- "step": 700
991
- },
992
- {
993
- "epoch": 0.12020460358056266,
994
- "grad_norm": 0.1284978687763214,
995
- "learning_rate": 7.498083019104527e-05,
996
- "loss": 0.013128276169300079,
997
- "step": 705
998
- },
999
- {
1000
- "epoch": 0.12105711849957375,
1001
- "grad_norm": 0.19205476343631744,
1002
- "learning_rate": 7.498000228965913e-05,
1003
- "loss": 0.014518238604068756,
1004
- "step": 710
1005
- },
1006
- {
1007
- "epoch": 0.12190963341858482,
1008
- "grad_norm": 0.25064221024513245,
1009
- "learning_rate": 7.497915689136119e-05,
1010
- "loss": 0.014235696196556092,
1011
- "step": 715
1012
- },
1013
- {
1014
- "epoch": 0.12276214833759591,
1015
- "grad_norm": 0.11866044998168945,
1016
- "learning_rate": 7.497829399654607e-05,
1017
- "loss": 0.013622967898845673,
1018
- "step": 720
1019
- },
1020
- {
1021
- "epoch": 0.12361466325660699,
1022
- "grad_norm": 0.1366252452135086,
1023
- "learning_rate": 7.49774136056166e-05,
1024
- "loss": 0.013592693209648132,
1025
- "step": 725
1026
- },
1027
- {
1028
- "epoch": 0.12446717817561807,
1029
- "grad_norm": 0.12722773849964142,
1030
- "learning_rate": 7.497651571898379e-05,
1031
- "loss": 0.02055167257785797,
1032
- "step": 730
1033
- },
1034
- {
1035
- "epoch": 0.12531969309462915,
1036
- "grad_norm": 0.07723517715930939,
1037
- "learning_rate": 7.49756003370668e-05,
1038
- "loss": 0.012377069890499115,
1039
- "step": 735
1040
- },
1041
- {
1042
- "epoch": 0.12617220801364024,
1043
- "grad_norm": 0.11081967502832413,
1044
- "learning_rate": 7.497466746029293e-05,
1045
- "loss": 0.013797640800476074,
1046
- "step": 740
1047
- },
1048
- {
1049
- "epoch": 0.12702472293265132,
1050
- "grad_norm": 0.1117677390575409,
1051
- "learning_rate": 7.497371708909771e-05,
1052
- "loss": 0.01338990479707718,
1053
- "step": 745
1054
- },
1055
- {
1056
- "epoch": 0.1278772378516624,
1057
- "grad_norm": 0.24659112095832825,
1058
- "learning_rate": 7.497274922392483e-05,
1059
- "loss": 0.012844511866569519,
1060
- "step": 750
1061
- },
1062
- {
1063
- "epoch": 0.1287297527706735,
1064
- "grad_norm": 0.24900244176387787,
1065
- "learning_rate": 7.497176386522606e-05,
1066
- "loss": 0.015656352043151855,
1067
- "step": 755
1068
- },
1069
- {
1070
- "epoch": 0.12958226768968456,
1071
- "grad_norm": 0.1331390142440796,
1072
- "learning_rate": 7.497076101346144e-05,
1073
- "loss": 0.013722085952758789,
1074
- "step": 760
1075
- },
1076
- {
1077
- "epoch": 0.13043478260869565,
1078
- "grad_norm": 0.1224697008728981,
1079
- "learning_rate": 7.496974066909913e-05,
1080
- "loss": 0.011186304688453674,
1081
- "step": 765
1082
- },
1083
- {
1084
- "epoch": 0.13128729752770674,
1085
- "grad_norm": 0.11053820699453354,
1086
- "learning_rate": 7.496870283261546e-05,
1087
- "loss": 0.012894454598426818,
1088
- "step": 770
1089
- },
1090
- {
1091
- "epoch": 0.13213981244671782,
1092
- "grad_norm": 0.09663277864456177,
1093
- "learning_rate": 7.49676475044949e-05,
1094
- "loss": 0.015331995487213135,
1095
- "step": 775
1096
- },
1097
- {
1098
- "epoch": 0.1329923273657289,
1099
- "grad_norm": 0.1483219712972641,
1100
- "learning_rate": 7.496657468523014e-05,
1101
- "loss": 0.015070399641990662,
1102
- "step": 780
1103
- },
1104
- {
1105
- "epoch": 0.13384484228473997,
1106
- "grad_norm": 0.12375539541244507,
1107
- "learning_rate": 7.496548437532202e-05,
1108
- "loss": 0.013722005486488342,
1109
- "step": 785
1110
- },
1111
- {
1112
- "epoch": 0.13469735720375106,
1113
- "grad_norm": 0.14662130177021027,
1114
- "learning_rate": 7.496437657527949e-05,
1115
- "loss": 0.012852996587753296,
1116
- "step": 790
1117
- },
1118
- {
1119
- "epoch": 0.13554987212276215,
1120
- "grad_norm": 0.1740300953388214,
1121
- "learning_rate": 7.496325128561975e-05,
1122
- "loss": 0.014796209335327149,
1123
- "step": 795
1124
- },
1125
- {
1126
- "epoch": 0.13640238704177324,
1127
- "grad_norm": 0.10670243203639984,
1128
- "learning_rate": 7.496210850686809e-05,
1129
- "loss": 0.013983005285263061,
1130
- "step": 800
1131
- },
1132
- {
1133
- "epoch": 0.13725490196078433,
1134
- "grad_norm": 0.1362515240907669,
1135
- "learning_rate": 7.496094823955801e-05,
1136
- "loss": 0.014472618699073792,
1137
- "step": 805
1138
- },
1139
- {
1140
- "epoch": 0.13810741687979539,
1141
- "grad_norm": 0.11403689533472061,
1142
- "learning_rate": 7.495977048423117e-05,
1143
- "loss": 0.01294848918914795,
1144
- "step": 810
1145
- },
1146
- {
1147
- "epoch": 0.13895993179880647,
1148
- "grad_norm": 0.11532565206289291,
1149
- "learning_rate": 7.495857524143738e-05,
1150
- "loss": 0.011346425861120224,
1151
- "step": 815
1152
- },
1153
- {
1154
- "epoch": 0.13981244671781756,
1155
- "grad_norm": 0.1506761759519577,
1156
- "learning_rate": 7.495736251173463e-05,
1157
- "loss": 0.016532811522483825,
1158
- "step": 820
1159
- },
1160
- {
1161
- "epoch": 0.14066496163682865,
1162
- "grad_norm": 0.10915899276733398,
1163
- "learning_rate": 7.495613229568903e-05,
1164
- "loss": 0.01278679072856903,
1165
- "step": 825
1166
- },
1167
- {
1168
- "epoch": 0.14151747655583974,
1169
- "grad_norm": 0.10394130647182465,
1170
- "learning_rate": 7.49548845938749e-05,
1171
- "loss": 0.011449626088142395,
1172
- "step": 830
1173
- },
1174
- {
1175
- "epoch": 0.1423699914748508,
1176
- "grad_norm": 0.08730677515268326,
1177
- "learning_rate": 7.495361940687475e-05,
1178
- "loss": 0.011916645616292954,
1179
- "step": 835
1180
- },
1181
- {
1182
- "epoch": 0.1432225063938619,
1183
- "grad_norm": 0.08257348835468292,
1184
- "learning_rate": 7.495233673527914e-05,
1185
- "loss": 0.013689276576042176,
1186
- "step": 840
1187
- },
1188
- {
1189
- "epoch": 0.14407502131287298,
1190
- "grad_norm": 0.182078555226326,
1191
- "learning_rate": 7.495103657968692e-05,
1192
- "loss": 0.019141729176044463,
1193
- "step": 845
1194
- },
1195
- {
1196
- "epoch": 0.14492753623188406,
1197
- "grad_norm": 0.105568528175354,
1198
- "learning_rate": 7.494971894070501e-05,
1199
- "loss": 0.015522226691246033,
1200
- "step": 850
1201
- },
1202
- {
1203
- "epoch": 0.14578005115089515,
1204
- "grad_norm": 0.09030122309923172,
1205
- "learning_rate": 7.494838381894856e-05,
1206
- "loss": 0.012900182604789734,
1207
- "step": 855
1208
- },
1209
- {
1210
- "epoch": 0.1466325660699062,
1211
- "grad_norm": 0.11199921369552612,
1212
- "learning_rate": 7.494703121504082e-05,
1213
- "loss": 0.013011230528354645,
1214
- "step": 860
1215
- },
1216
- {
1217
- "epoch": 0.1474850809889173,
1218
- "grad_norm": 0.11342430859804153,
1219
- "learning_rate": 7.494566112961325e-05,
1220
- "loss": 0.015477502346038818,
1221
- "step": 865
1222
- },
1223
- {
1224
- "epoch": 0.1483375959079284,
1225
- "grad_norm": 0.07956613600254059,
1226
- "learning_rate": 7.494427356330544e-05,
1227
- "loss": 0.011270551383495331,
1228
- "step": 870
1229
- },
1230
- {
1231
- "epoch": 0.14919011082693948,
1232
- "grad_norm": 0.13356897234916687,
1233
- "learning_rate": 7.494286851676515e-05,
1234
- "loss": 0.012481572479009629,
1235
- "step": 875
1236
- },
1237
- {
1238
- "epoch": 0.15004262574595056,
1239
- "grad_norm": 0.14493827521800995,
1240
- "learning_rate": 7.494144599064833e-05,
1241
- "loss": 0.015318951010704041,
1242
- "step": 880
1243
- },
1244
- {
1245
- "epoch": 0.15089514066496162,
1246
- "grad_norm": 0.13254614174365997,
1247
- "learning_rate": 7.494000598561902e-05,
1248
- "loss": 0.012697519361972808,
1249
- "step": 885
1250
- },
1251
- {
1252
- "epoch": 0.1517476555839727,
1253
- "grad_norm": 0.12164720892906189,
1254
- "learning_rate": 7.49385485023495e-05,
1255
- "loss": 0.011361779272556305,
1256
- "step": 890
1257
- },
1258
- {
1259
- "epoch": 0.1526001705029838,
1260
- "grad_norm": 0.10743863880634308,
1261
- "learning_rate": 7.493707354152015e-05,
1262
- "loss": 0.012824115157127381,
1263
- "step": 895
1264
- },
1265
- {
1266
- "epoch": 0.1534526854219949,
1267
- "grad_norm": 0.0933203473687172,
1268
- "learning_rate": 7.493558110381954e-05,
1269
- "loss": 0.012234293669462205,
1270
- "step": 900
1271
- },
1272
- {
1273
- "epoch": 0.15430520034100598,
1274
- "grad_norm": 0.10252948850393295,
1275
- "learning_rate": 7.493407118994437e-05,
1276
- "loss": 0.01874733567237854,
1277
- "step": 905
1278
- },
1279
- {
1280
- "epoch": 0.15515771526001704,
1281
- "grad_norm": 0.1184248998761177,
1282
- "learning_rate": 7.493254380059954e-05,
1283
- "loss": 0.012014241516590118,
1284
- "step": 910
1285
- },
1286
- {
1287
- "epoch": 0.15601023017902813,
1288
- "grad_norm": 0.10140799731016159,
1289
- "learning_rate": 7.49309989364981e-05,
1290
- "loss": 0.013288506865501403,
1291
- "step": 915
1292
- },
1293
- {
1294
- "epoch": 0.1568627450980392,
1295
- "grad_norm": 0.15704941749572754,
1296
- "learning_rate": 7.492943659836121e-05,
1297
- "loss": 0.012907981872558594,
1298
- "step": 920
1299
- },
1300
- {
1301
- "epoch": 0.1577152600170503,
1302
- "grad_norm": 0.10368761420249939,
1303
- "learning_rate": 7.492785678691822e-05,
1304
- "loss": 0.01384827345609665,
1305
- "step": 925
1306
- },
1307
- {
1308
- "epoch": 0.1585677749360614,
1309
- "grad_norm": 0.13517284393310547,
1310
- "learning_rate": 7.492625950290668e-05,
1311
- "loss": 0.014644764363765717,
1312
- "step": 930
1313
- },
1314
- {
1315
- "epoch": 0.15942028985507245,
1316
- "grad_norm": 0.08189263194799423,
1317
- "learning_rate": 7.492464474707222e-05,
1318
- "loss": 0.011974713951349258,
1319
- "step": 935
1320
- },
1321
- {
1322
- "epoch": 0.16027280477408354,
1323
- "grad_norm": 0.11737149208784103,
1324
- "learning_rate": 7.492301252016867e-05,
1325
- "loss": 0.014023615419864655,
1326
- "step": 940
1327
- },
1328
- {
1329
- "epoch": 0.16112531969309463,
1330
- "grad_norm": 0.15778031945228577,
1331
- "learning_rate": 7.492136282295801e-05,
1332
- "loss": 0.01533726304769516,
1333
- "step": 945
1334
- },
1335
- {
1336
- "epoch": 0.16197783461210571,
1337
- "grad_norm": 0.14194118976593018,
1338
- "learning_rate": 7.491969565621036e-05,
1339
- "loss": 0.01569782793521881,
1340
- "step": 950
1341
- },
1342
- {
1343
- "epoch": 0.1628303495311168,
1344
- "grad_norm": 0.12579558789730072,
1345
- "learning_rate": 7.491801102070403e-05,
1346
- "loss": 0.012844063341617584,
1347
- "step": 955
1348
- },
1349
- {
1350
- "epoch": 0.1636828644501279,
1351
- "grad_norm": 0.10578597337007523,
1352
- "learning_rate": 7.491630891722547e-05,
1353
- "loss": 0.011186198890209198,
1354
- "step": 960
1355
- },
1356
- {
1357
- "epoch": 0.16453537936913895,
1358
- "grad_norm": 0.12398207187652588,
1359
- "learning_rate": 7.491458934656925e-05,
1360
- "loss": 0.014328254759311676,
1361
- "step": 965
1362
- },
1363
- {
1364
- "epoch": 0.16538789428815004,
1365
- "grad_norm": 0.09390626102685928,
1366
- "learning_rate": 7.491285230953814e-05,
1367
- "loss": 0.013881708681583404,
1368
- "step": 970
1369
- },
1370
- {
1371
- "epoch": 0.16624040920716113,
1372
- "grad_norm": 0.17196106910705566,
1373
- "learning_rate": 7.491109780694303e-05,
1374
- "loss": 0.014424234628677368,
1375
- "step": 975
1376
- },
1377
- {
1378
- "epoch": 0.16709292412617222,
1379
- "grad_norm": 0.06994624435901642,
1380
- "learning_rate": 7.490932583960302e-05,
1381
- "loss": 0.010434426367282867,
1382
- "step": 980
1383
- },
1384
- {
1385
- "epoch": 0.1679454390451833,
1386
- "grad_norm": 0.09414499998092651,
1387
- "learning_rate": 7.490753640834527e-05,
1388
- "loss": 0.010483803600072861,
1389
- "step": 985
1390
- },
1391
- {
1392
- "epoch": 0.16879795396419436,
1393
- "grad_norm": 0.11002875864505768,
1394
- "learning_rate": 7.490572951400518e-05,
1395
- "loss": 0.009266073256731034,
1396
- "step": 990
1397
- },
1398
- {
1399
- "epoch": 0.16965046888320545,
1400
- "grad_norm": 0.14956022799015045,
1401
- "learning_rate": 7.490390515742626e-05,
1402
- "loss": 0.015529079735279084,
1403
- "step": 995
1404
- },
1405
- {
1406
- "epoch": 0.17050298380221654,
1407
- "grad_norm": 0.13828666508197784,
1408
- "learning_rate": 7.490206333946019e-05,
1409
- "loss": 0.011511271446943283,
1410
- "step": 1000
1411
- },
1412
- {
1413
- "epoch": 0.17135549872122763,
1414
- "grad_norm": 0.1265997439622879,
1415
- "learning_rate": 7.490020406096678e-05,
1416
- "loss": 0.010465707629919052,
1417
- "step": 1005
1418
- },
1419
- {
1420
- "epoch": 0.17220801364023872,
1421
- "grad_norm": 0.07772056013345718,
1422
- "learning_rate": 7.489832732281401e-05,
1423
- "loss": 0.013828210532665253,
1424
- "step": 1010
1425
- },
1426
- {
1427
- "epoch": 0.17306052855924978,
1428
- "grad_norm": 0.08055376261472702,
1429
- "learning_rate": 7.489643312587799e-05,
1430
- "loss": 0.013010218739509583,
1431
- "step": 1015
1432
- },
1433
- {
1434
- "epoch": 0.17391304347826086,
1435
- "grad_norm": 0.09059367328882217,
1436
- "learning_rate": 7.489452147104301e-05,
1437
- "loss": 0.013864235579967498,
1438
- "step": 1020
1439
- },
1440
- {
1441
- "epoch": 0.17476555839727195,
1442
- "grad_norm": 0.15273553133010864,
1443
- "learning_rate": 7.489259235920149e-05,
1444
- "loss": 0.013432112336158753,
1445
- "step": 1025
1446
- },
1447
- {
1448
- "epoch": 0.17561807331628304,
1449
- "grad_norm": 0.11343793570995331,
1450
- "learning_rate": 7.489064579125399e-05,
1451
- "loss": 0.01213686615228653,
1452
- "step": 1030
1453
- },
1454
- {
1455
- "epoch": 0.17647058823529413,
1456
- "grad_norm": 0.11880069226026535,
1457
- "learning_rate": 7.488868176810926e-05,
1458
- "loss": 0.012188264727592468,
1459
- "step": 1035
1460
- },
1461
- {
1462
- "epoch": 0.1773231031543052,
1463
- "grad_norm": 0.16270127892494202,
1464
- "learning_rate": 7.488670029068415e-05,
1465
- "loss": 0.015294317901134492,
1466
- "step": 1040
1467
- },
1468
- {
1469
- "epoch": 0.17817561807331628,
1470
- "grad_norm": 0.11542797088623047,
1471
- "learning_rate": 7.488470135990369e-05,
1472
- "loss": 0.013500772416591644,
1473
- "step": 1045
1474
- },
1475
- {
1476
- "epoch": 0.17902813299232737,
1477
- "grad_norm": 0.09579882025718689,
1478
- "learning_rate": 7.488268497670103e-05,
1479
- "loss": 0.013453315198421478,
1480
- "step": 1050
1481
- },
1482
- {
1483
- "epoch": 0.17988064791133845,
1484
- "grad_norm": 0.08847666531801224,
1485
- "learning_rate": 7.488065114201752e-05,
1486
- "loss": 0.0153861403465271,
1487
- "step": 1055
1488
- },
1489
- {
1490
- "epoch": 0.18073316283034954,
1491
- "grad_norm": 0.11167823523283005,
1492
- "learning_rate": 7.487859985680257e-05,
1493
- "loss": 0.011502107977867127,
1494
- "step": 1060
1495
- },
1496
- {
1497
- "epoch": 0.1815856777493606,
1498
- "grad_norm": 0.11255413293838501,
1499
- "learning_rate": 7.487653112201385e-05,
1500
- "loss": 0.012194579839706421,
1501
- "step": 1065
1502
- },
1503
- {
1504
- "epoch": 0.1824381926683717,
1505
- "grad_norm": 0.10929680615663528,
1506
- "learning_rate": 7.487444493861705e-05,
1507
- "loss": 0.015874122083187104,
1508
- "step": 1070
1509
- },
1510
- {
1511
- "epoch": 0.18329070758738278,
1512
- "grad_norm": 0.10753592848777771,
1513
- "learning_rate": 7.487234130758613e-05,
1514
- "loss": 0.009445396810770034,
1515
- "step": 1075
1516
- },
1517
- {
1518
- "epoch": 0.18414322250639387,
1519
- "grad_norm": 0.11368737369775772,
1520
- "learning_rate": 7.487022022990309e-05,
1521
- "loss": 0.011674505472183228,
1522
- "step": 1080
1523
- },
1524
- {
1525
- "epoch": 0.18499573742540495,
1526
- "grad_norm": 0.12944836914539337,
1527
- "learning_rate": 7.486808170655813e-05,
1528
- "loss": 0.011856313049793243,
1529
- "step": 1085
1530
- },
1531
- {
1532
- "epoch": 0.18584825234441602,
1533
- "grad_norm": 0.0833214819431305,
1534
- "learning_rate": 7.48659257385496e-05,
1535
- "loss": 0.014119544625282287,
1536
- "step": 1090
1537
- },
1538
- {
1539
- "epoch": 0.1867007672634271,
1540
- "grad_norm": 0.11955790221691132,
1541
- "learning_rate": 7.486375232688397e-05,
1542
- "loss": 0.012977911531925202,
1543
- "step": 1095
1544
- },
1545
- {
1546
- "epoch": 0.1875532821824382,
1547
- "grad_norm": 0.1452455222606659,
1548
- "learning_rate": 7.486156147257584e-05,
1549
- "loss": 0.015410827100276947,
1550
- "step": 1100
1551
- },
1552
- {
1553
- "epoch": 0.18840579710144928,
1554
- "grad_norm": 0.09630092978477478,
1555
- "learning_rate": 7.485935317664801e-05,
1556
- "loss": 0.013698874413967133,
1557
- "step": 1105
1558
- },
1559
- {
1560
- "epoch": 0.18925831202046037,
1561
- "grad_norm": 0.11687257140874863,
1562
- "learning_rate": 7.485712744013137e-05,
1563
- "loss": 0.013196484744548797,
1564
- "step": 1110
1565
- },
1566
- {
1567
- "epoch": 0.19011082693947143,
1568
- "grad_norm": 0.08894632011651993,
1569
- "learning_rate": 7.485488426406495e-05,
1570
- "loss": 0.01540881097316742,
1571
- "step": 1115
1572
- },
1573
- {
1574
- "epoch": 0.19096334185848252,
1575
- "grad_norm": 0.09196458756923676,
1576
- "learning_rate": 7.485262364949597e-05,
1577
- "loss": 0.012018527090549468,
1578
- "step": 1120
1579
- },
1580
- {
1581
- "epoch": 0.1918158567774936,
1582
- "grad_norm": 0.12328553944826126,
1583
- "learning_rate": 7.485034559747974e-05,
1584
- "loss": 0.014925773441791534,
1585
- "step": 1125
1586
- },
1587
- {
1588
- "epoch": 0.1926683716965047,
1589
- "grad_norm": 0.12566202878952026,
1590
- "learning_rate": 7.484805010907975e-05,
1591
- "loss": 0.01104198843240738,
1592
- "step": 1130
1593
- },
1594
- {
1595
- "epoch": 0.19352088661551578,
1596
- "grad_norm": 0.1022765263915062,
1597
- "learning_rate": 7.484573718536758e-05,
1598
- "loss": 0.0118367500603199,
1599
- "step": 1135
1600
- },
1601
- {
1602
- "epoch": 0.19437340153452684,
1603
- "grad_norm": 0.09682224690914154,
1604
- "learning_rate": 7.4843406827423e-05,
1605
- "loss": 0.011423001438379288,
1606
- "step": 1140
1607
- },
1608
- {
1609
- "epoch": 0.19522591645353793,
1610
- "grad_norm": 0.15166254341602325,
1611
- "learning_rate": 7.484105903633388e-05,
1612
- "loss": 0.014048118889331818,
1613
- "step": 1145
1614
- },
1615
- {
1616
- "epoch": 0.19607843137254902,
1617
- "grad_norm": 0.09285032004117966,
1618
- "learning_rate": 7.483869381319627e-05,
1619
- "loss": 0.012882034480571746,
1620
- "step": 1150
1621
- },
1622
- {
1623
- "epoch": 0.1969309462915601,
1624
- "grad_norm": 0.09011214971542358,
1625
- "learning_rate": 7.483631115911434e-05,
1626
- "loss": 0.01419239491224289,
1627
- "step": 1155
1628
- },
1629
- {
1630
- "epoch": 0.1977834612105712,
1631
- "grad_norm": 0.14540383219718933,
1632
- "learning_rate": 7.483391107520037e-05,
1633
- "loss": 0.014623096585273743,
1634
- "step": 1160
1635
- },
1636
- {
1637
- "epoch": 0.19863597612958228,
1638
- "grad_norm": 0.12326448410749435,
1639
- "learning_rate": 7.483149356257479e-05,
1640
- "loss": 0.013109591603279114,
1641
- "step": 1165
1642
- },
1643
- {
1644
- "epoch": 0.19948849104859334,
1645
- "grad_norm": 0.07067535817623138,
1646
- "learning_rate": 7.482905862236622e-05,
1647
- "loss": 0.013740380108356477,
1648
- "step": 1170
1649
- },
1650
- {
1651
- "epoch": 0.20034100596760443,
1652
- "grad_norm": 0.10170529782772064,
1653
- "learning_rate": 7.482660625571134e-05,
1654
- "loss": 0.011151721328496933,
1655
- "step": 1175
1656
- },
1657
- {
1658
- "epoch": 0.20119352088661552,
1659
- "grad_norm": 0.10074683278799057,
1660
- "learning_rate": 7.482413646375498e-05,
1661
- "loss": 0.01180294007062912,
1662
- "step": 1180
1663
- },
1664
- {
1665
- "epoch": 0.2020460358056266,
1666
- "grad_norm": 0.047992780804634094,
1667
- "learning_rate": 7.482164924765016e-05,
1668
- "loss": 0.01065710186958313,
1669
- "step": 1185
1670
- },
1671
- {
1672
- "epoch": 0.2028985507246377,
1673
- "grad_norm": 0.1279197484254837,
1674
- "learning_rate": 7.481914460855796e-05,
1675
- "loss": 0.012219739705324173,
1676
- "step": 1190
1677
- },
1678
- {
1679
- "epoch": 0.20375106564364875,
1680
- "grad_norm": 0.11339649558067322,
1681
- "learning_rate": 7.481662254764765e-05,
1682
- "loss": 0.012664712965488434,
1683
- "step": 1195
1684
- },
1685
- {
1686
- "epoch": 0.20460358056265984,
1687
- "grad_norm": 0.10576959699392319,
1688
- "learning_rate": 7.481408306609662e-05,
1689
- "loss": 0.010009048134088516,
1690
- "step": 1200
1691
- },
1692
- {
1693
- "epoch": 0.20545609548167093,
1694
- "grad_norm": 0.08073283731937408,
1695
- "learning_rate": 7.481152616509037e-05,
1696
- "loss": 0.011126396805047989,
1697
- "step": 1205
1698
- },
1699
- {
1700
- "epoch": 0.20630861040068202,
1701
- "grad_norm": 0.09888558834791183,
1702
- "learning_rate": 7.480895184582253e-05,
1703
- "loss": 0.013096305727958679,
1704
- "step": 1210
1705
- },
1706
- {
1707
- "epoch": 0.2071611253196931,
1708
- "grad_norm": 0.09703118354082108,
1709
- "learning_rate": 7.48063601094949e-05,
1710
- "loss": 0.010653502494096755,
1711
- "step": 1215
1712
- },
1713
- {
1714
- "epoch": 0.20801364023870417,
1715
- "grad_norm": 0.10693266987800598,
1716
- "learning_rate": 7.48037509573174e-05,
1717
- "loss": 0.012283077836036682,
1718
- "step": 1220
1719
- },
1720
- {
1721
- "epoch": 0.20886615515771526,
1722
- "grad_norm": 0.1024889275431633,
1723
- "learning_rate": 7.480112439050804e-05,
1724
- "loss": 0.012971282005310059,
1725
- "step": 1225
1726
- },
1727
- {
1728
- "epoch": 0.20971867007672634,
1729
- "grad_norm": 0.10043203830718994,
1730
- "learning_rate": 7.4798480410293e-05,
1731
- "loss": 0.011451539397239686,
1732
- "step": 1230
1733
- },
1734
- {
1735
- "epoch": 0.21057118499573743,
1736
- "grad_norm": 0.11248672753572464,
1737
- "learning_rate": 7.47958190179066e-05,
1738
- "loss": 0.012805460393428803,
1739
- "step": 1235
1740
- },
1741
- {
1742
- "epoch": 0.21142369991474852,
1743
- "grad_norm": 0.08651525527238846,
1744
- "learning_rate": 7.479314021459123e-05,
1745
- "loss": 0.013016811013221741,
1746
- "step": 1240
1747
- },
1748
- {
1749
- "epoch": 0.21227621483375958,
1750
- "grad_norm": 0.06062095984816551,
1751
- "learning_rate": 7.479044400159746e-05,
1752
- "loss": 0.01299230456352234,
1753
- "step": 1245
1754
- },
1755
- {
1756
- "epoch": 0.21312872975277067,
1757
- "grad_norm": 0.1589215248823166,
1758
- "learning_rate": 7.478773038018397e-05,
1759
- "loss": 0.012607543170452118,
1760
- "step": 1250
1761
- },
1762
- {
1763
- "epoch": 0.21398124467178176,
1764
- "grad_norm": 0.12105076014995575,
1765
- "learning_rate": 7.478499935161758e-05,
1766
- "loss": 0.012772174179553985,
1767
- "step": 1255
1768
- },
1769
- {
1770
- "epoch": 0.21483375959079284,
1771
- "grad_norm": 0.0846070721745491,
1772
- "learning_rate": 7.478225091717323e-05,
1773
- "loss": 0.009387130290269852,
1774
- "step": 1260
1775
- },
1776
- {
1777
- "epoch": 0.21568627450980393,
1778
- "grad_norm": 0.054560381919145584,
1779
- "learning_rate": 7.477948507813396e-05,
1780
- "loss": 0.013299009203910828,
1781
- "step": 1265
1782
- },
1783
- {
1784
- "epoch": 0.216538789428815,
1785
- "grad_norm": 0.10125566273927689,
1786
- "learning_rate": 7.477670183579094e-05,
1787
- "loss": 0.013010343909263611,
1788
- "step": 1270
1789
- },
1790
- {
1791
- "epoch": 0.21739130434782608,
1792
- "grad_norm": 0.10493458807468414,
1793
- "learning_rate": 7.477390119144353e-05,
1794
- "loss": 0.013531042635440827,
1795
- "step": 1275
1796
- },
1797
- {
1798
- "epoch": 0.21824381926683717,
1799
- "grad_norm": 0.07453935593366623,
1800
- "learning_rate": 7.477108314639913e-05,
1801
- "loss": 0.01330820918083191,
1802
- "step": 1280
1803
- },
1804
- {
1805
- "epoch": 0.21909633418584826,
1806
- "grad_norm": 0.14430643618106842,
1807
- "learning_rate": 7.47682477019733e-05,
1808
- "loss": 0.015061555802822113,
1809
- "step": 1285
1810
- },
1811
- {
1812
- "epoch": 0.21994884910485935,
1813
- "grad_norm": 0.1073407456278801,
1814
- "learning_rate": 7.476539485948973e-05,
1815
- "loss": 0.011137319356203079,
1816
- "step": 1290
1817
- },
1818
- {
1819
- "epoch": 0.2208013640238704,
1820
- "grad_norm": 0.16794899106025696,
1821
- "learning_rate": 7.476252462028021e-05,
1822
- "loss": 0.013223762810230254,
1823
- "step": 1295
1824
- },
1825
- {
1826
- "epoch": 0.2216538789428815,
1827
- "grad_norm": 0.11509175598621368,
1828
- "learning_rate": 7.475963698568468e-05,
1829
- "loss": 0.012790821492671967,
1830
- "step": 1300
1831
- },
1832
- {
1833
- "epoch": 0.22250639386189258,
1834
- "grad_norm": 0.08615118265151978,
1835
- "learning_rate": 7.475673195705116e-05,
1836
- "loss": 0.011050455272197723,
1837
- "step": 1305
1838
- },
1839
- {
1840
- "epoch": 0.22335890878090367,
1841
- "grad_norm": 0.1186874732375145,
1842
- "learning_rate": 7.475380953573583e-05,
1843
- "loss": 0.011253353953361512,
1844
- "step": 1310
1845
- },
1846
- {
1847
- "epoch": 0.22421142369991476,
1848
- "grad_norm": 0.08680208027362823,
1849
- "learning_rate": 7.475086972310297e-05,
1850
- "loss": 0.010736887156963349,
1851
- "step": 1315
1852
- },
1853
- {
1854
- "epoch": 0.22506393861892582,
1855
- "grad_norm": 0.08204677700996399,
1856
- "learning_rate": 7.474791252052498e-05,
1857
- "loss": 0.011695361882448196,
1858
- "step": 1320
1859
- },
1860
- {
1861
- "epoch": 0.2259164535379369,
1862
- "grad_norm": 0.08945680409669876,
1863
- "learning_rate": 7.47449379293824e-05,
1864
- "loss": 0.01684057414531708,
1865
- "step": 1325
1866
- },
1867
- {
1868
- "epoch": 0.226768968456948,
1869
- "grad_norm": 0.12350911647081375,
1870
- "learning_rate": 7.474194595106384e-05,
1871
- "loss": 0.012560060620307923,
1872
- "step": 1330
1873
- },
1874
- {
1875
- "epoch": 0.22762148337595908,
1876
- "grad_norm": 0.09201455116271973,
1877
- "learning_rate": 7.473893658696605e-05,
1878
- "loss": 0.01128845140337944,
1879
- "step": 1335
1880
- },
1881
- {
1882
- "epoch": 0.22847399829497017,
1883
- "grad_norm": 0.12938448786735535,
1884
- "learning_rate": 7.473590983849396e-05,
1885
- "loss": 0.011510471999645232,
1886
- "step": 1340
1887
- },
1888
- {
1889
- "epoch": 0.22932651321398123,
1890
- "grad_norm": 0.0837291032075882,
1891
- "learning_rate": 7.473286570706047e-05,
1892
- "loss": 0.011677465587854385,
1893
- "step": 1345
1894
- },
1895
- {
1896
- "epoch": 0.23017902813299232,
1897
- "grad_norm": 0.12514546513557434,
1898
- "learning_rate": 7.472980419408675e-05,
1899
- "loss": 0.01308433711528778,
1900
- "step": 1350
1901
- },
1902
- {
1903
- "epoch": 0.2310315430520034,
1904
- "grad_norm": 0.05483829975128174,
1905
- "learning_rate": 7.472672530100199e-05,
1906
- "loss": 0.007203156501054764,
1907
- "step": 1355
1908
- },
1909
- {
1910
- "epoch": 0.2318840579710145,
1911
- "grad_norm": 0.13903425633907318,
1912
- "learning_rate": 7.472362902924352e-05,
1913
- "loss": 0.013231572508811951,
1914
- "step": 1360
1915
- },
1916
- {
1917
- "epoch": 0.23273657289002558,
1918
- "grad_norm": 0.09622041881084442,
1919
- "learning_rate": 7.472051538025678e-05,
1920
- "loss": 0.013325902819633483,
1921
- "step": 1365
1922
- },
1923
- {
1924
- "epoch": 0.23358908780903667,
1925
- "grad_norm": 0.0792275071144104,
1926
- "learning_rate": 7.471738435549533e-05,
1927
- "loss": 0.011098403483629227,
1928
- "step": 1370
1929
- },
1930
- {
1931
- "epoch": 0.23444160272804773,
1932
- "grad_norm": 0.11833506077528,
1933
- "learning_rate": 7.471423595642084e-05,
1934
- "loss": 0.014845246076583862,
1935
- "step": 1375
1936
- },
1937
- {
1938
- "epoch": 0.23529411764705882,
1939
- "grad_norm": 0.07531571388244629,
1940
- "learning_rate": 7.471107018450309e-05,
1941
- "loss": 0.011498528718948364,
1942
- "step": 1380
1943
- },
1944
- {
1945
- "epoch": 0.2361466325660699,
1946
- "grad_norm": 0.08739249408245087,
1947
- "learning_rate": 7.470788704121995e-05,
1948
- "loss": 0.013241058588027954,
1949
- "step": 1385
1950
- },
1951
- {
1952
- "epoch": 0.236999147485081,
1953
- "grad_norm": 0.07113732397556305,
1954
- "learning_rate": 7.470468652805743e-05,
1955
- "loss": 0.009632241725921632,
1956
- "step": 1390
1957
- },
1958
- {
1959
- "epoch": 0.23785166240409208,
1960
- "grad_norm": 0.07838430255651474,
1961
- "learning_rate": 7.470146864650965e-05,
1962
- "loss": 0.009344339370727539,
1963
- "step": 1395
1964
- },
1965
- {
1966
- "epoch": 0.23870417732310314,
1967
- "grad_norm": 0.1086372509598732,
1968
- "learning_rate": 7.46982333980788e-05,
1969
- "loss": 0.014708395302295684,
1970
- "step": 1400
1971
- },
1972
- {
1973
- "epoch": 0.23955669224211423,
1974
- "grad_norm": 0.06628046184778214,
1975
- "learning_rate": 7.469498078427519e-05,
1976
- "loss": 0.011472882330417633,
1977
- "step": 1405
1978
- },
1979
- {
1980
- "epoch": 0.24040920716112532,
1981
- "grad_norm": 0.13003386557102203,
1982
- "learning_rate": 7.46917108066173e-05,
1983
- "loss": 0.011933800578117371,
1984
- "step": 1410
1985
- },
1986
- {
1987
- "epoch": 0.2412617220801364,
1988
- "grad_norm": 0.07365831732749939,
1989
- "learning_rate": 7.468842346663162e-05,
1990
- "loss": 0.0102902352809906,
1991
- "step": 1415
1992
- },
1993
- {
1994
- "epoch": 0.2421142369991475,
1995
- "grad_norm": 0.11540158838033676,
1996
- "learning_rate": 7.468511876585279e-05,
1997
- "loss": 0.012016136944293977,
1998
- "step": 1420
1999
- },
2000
- {
2001
- "epoch": 0.24296675191815856,
2002
- "grad_norm": 0.11687944084405899,
2003
- "learning_rate": 7.468179670582359e-05,
2004
- "loss": 0.01773642897605896,
2005
- "step": 1425
2006
- },
2007
- {
2008
- "epoch": 0.24381926683716965,
2009
- "grad_norm": 0.10464506596326828,
2010
- "learning_rate": 7.467845728809483e-05,
2011
- "loss": 0.009476778656244278,
2012
- "step": 1430
2013
- },
2014
- {
2015
- "epoch": 0.24467178175618073,
2016
- "grad_norm": 0.11479093879461288,
2017
- "learning_rate": 7.46751005142255e-05,
2018
- "loss": 0.014100676774978638,
2019
- "step": 1435
2020
- },
2021
- {
2022
- "epoch": 0.24552429667519182,
2023
- "grad_norm": 0.1388610154390335,
2024
- "learning_rate": 7.46717263857826e-05,
2025
- "loss": 0.012735754251480103,
2026
- "step": 1440
2027
- },
2028
- {
2029
- "epoch": 0.2463768115942029,
2030
- "grad_norm": 0.08022485673427582,
2031
- "learning_rate": 7.466833490434132e-05,
2032
- "loss": 0.014391189813613892,
2033
- "step": 1445
2034
- },
2035
- {
2036
- "epoch": 0.24722932651321397,
2037
- "grad_norm": 0.12174725532531738,
2038
- "learning_rate": 7.466492607148492e-05,
2039
- "loss": 0.013387931883335114,
2040
- "step": 1450
2041
- },
2042
- {
2043
- "epoch": 0.24808184143222506,
2044
- "grad_norm": 0.1102161779999733,
2045
- "learning_rate": 7.466149988880474e-05,
2046
- "loss": 0.01143224760890007,
2047
- "step": 1455
2048
- },
2049
- {
2050
- "epoch": 0.24893435635123615,
2051
- "grad_norm": 0.14878468215465546,
2052
- "learning_rate": 7.465805635790024e-05,
2053
- "loss": 0.010428830236196517,
2054
- "step": 1460
2055
- },
2056
- {
2057
- "epoch": 0.24978687127024723,
2058
- "grad_norm": 0.3046579658985138,
2059
- "learning_rate": 7.4654595480379e-05,
2060
- "loss": 0.012648795545101166,
2061
- "step": 1465
2062
- },
2063
- {
2064
- "epoch": 0.24995737425404946,
2065
- "eval_loss": 0.036103956401348114,
2066
- "eval_runtime": 3.6524,
2067
- "eval_samples_per_second": 68.995,
2068
- "eval_steps_per_second": 1.095,
2069
- "step": 1466
2070
- },
2071
- {
2072
- "eval_cer_subset": 0.01393977885257381,
2073
- "eval_cer_subset_edit_distance": 856,
2074
- "eval_cer_subset_groups": 250,
2075
- "eval_cer_subset_items": 250,
2076
- "eval_cer_subset_ref_chars": 61407,
2077
- "step": 1466
2078
- }
2079
- ],
2080
- "logging_steps": 5,
2081
- "max_steps": 23460,
2082
- "num_input_tokens_seen": 0,
2083
- "num_train_epochs": 4,
2084
- "save_steps": 2932,
2085
- "stateful_callbacks": {
2086
- "TrainerControl": {
2087
- "args": {
2088
- "should_epoch_stop": false,
2089
- "should_evaluate": false,
2090
- "should_log": false,
2091
- "should_save": true,
2092
- "should_training_stop": false
2093
- },
2094
- "attributes": {}
2095
- }
2096
- },
2097
- "total_flos": 4.941642367710904e+17,
2098
- "train_batch_size": 32,
2099
- "trial_name": null,
2100
- "trial_params": null
2101
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-1466/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6adec376d54028ef57ef3dc856a5cba12bab9c0d580369637fa983b6072064f7
3
- size 5841
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/README.md DELETED
@@ -1,210 +0,0 @@
1
- ---
2
- base_model: unsloth/Meta-Llama-3.1-8B
3
- library_name: peft
4
- pipeline_tag: text-generation
5
- tags:
6
- - base_model:adapter:unsloth/Meta-Llama-3.1-8B
7
- - lora
8
- - sft
9
- - transformers
10
- - trl
11
- - unsloth
12
- ---
13
-
14
- # Model Card for Model ID
15
-
16
- <!-- Provide a quick summary of what the model is/does. -->
17
-
18
-
19
-
20
- ## Model Details
21
-
22
- ### Model Description
23
-
24
- <!-- Provide a longer summary of what this model is. -->
25
-
26
-
27
-
28
- - **Developed by:** [More Information Needed]
29
- - **Funded by [optional]:** [More Information Needed]
30
- - **Shared by [optional]:** [More Information Needed]
31
- - **Model type:** [More Information Needed]
32
- - **Language(s) (NLP):** [More Information Needed]
33
- - **License:** [More Information Needed]
34
- - **Finetuned from model [optional]:** [More Information Needed]
35
-
36
- ### Model Sources [optional]
37
-
38
- <!-- Provide the basic links for the model. -->
39
-
40
- - **Repository:** [More Information Needed]
41
- - **Paper [optional]:** [More Information Needed]
42
- - **Demo [optional]:** [More Information Needed]
43
-
44
- ## Uses
45
-
46
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
-
48
- ### Direct Use
49
-
50
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
-
52
- [More Information Needed]
53
-
54
- ### Downstream Use [optional]
55
-
56
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
-
58
- [More Information Needed]
59
-
60
- ### Out-of-Scope Use
61
-
62
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
-
64
- [More Information Needed]
65
-
66
- ## Bias, Risks, and Limitations
67
-
68
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
-
70
- [More Information Needed]
71
-
72
- ### Recommendations
73
-
74
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
-
76
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
-
78
- ## How to Get Started with the Model
79
-
80
- Use the code below to get started with the model.
81
-
82
- [More Information Needed]
83
-
84
- ## Training Details
85
-
86
- ### Training Data
87
-
88
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
-
90
- [More Information Needed]
91
-
92
- ### Training Procedure
93
-
94
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
-
96
- #### Preprocessing [optional]
97
-
98
- [More Information Needed]
99
-
100
-
101
- #### Training Hyperparameters
102
-
103
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
-
105
- #### Speeds, Sizes, Times [optional]
106
-
107
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
-
109
- [More Information Needed]
110
-
111
- ## Evaluation
112
-
113
- <!-- This section describes the evaluation protocols and provides the results. -->
114
-
115
- ### Testing Data, Factors & Metrics
116
-
117
- #### Testing Data
118
-
119
- <!-- This should link to a Dataset Card if possible. -->
120
-
121
- [More Information Needed]
122
-
123
- #### Factors
124
-
125
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
-
127
- [More Information Needed]
128
-
129
- #### Metrics
130
-
131
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
-
133
- [More Information Needed]
134
-
135
- ### Results
136
-
137
- [More Information Needed]
138
-
139
- #### Summary
140
-
141
-
142
-
143
- ## Model Examination [optional]
144
-
145
- <!-- Relevant interpretability work for the model goes here -->
146
-
147
- [More Information Needed]
148
-
149
- ## Environmental Impact
150
-
151
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
-
153
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
-
155
- - **Hardware Type:** [More Information Needed]
156
- - **Hours used:** [More Information Needed]
157
- - **Cloud Provider:** [More Information Needed]
158
- - **Compute Region:** [More Information Needed]
159
- - **Carbon Emitted:** [More Information Needed]
160
-
161
- ## Technical Specifications [optional]
162
-
163
- ### Model Architecture and Objective
164
-
165
- [More Information Needed]
166
-
167
- ### Compute Infrastructure
168
-
169
- [More Information Needed]
170
-
171
- #### Hardware
172
-
173
- [More Information Needed]
174
-
175
- #### Software
176
-
177
- [More Information Needed]
178
-
179
- ## Citation [optional]
180
-
181
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
-
183
- **BibTeX:**
184
-
185
- [More Information Needed]
186
-
187
- **APA:**
188
-
189
- [More Information Needed]
190
-
191
- ## Glossary [optional]
192
-
193
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
-
195
- [More Information Needed]
196
-
197
- ## More Information [optional]
198
-
199
- [More Information Needed]
200
-
201
- ## Model Card Authors [optional]
202
-
203
- [More Information Needed]
204
-
205
- ## Model Card Contact
206
-
207
- [More Information Needed]
208
- ### Framework versions
209
-
210
- - PEFT 0.18.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/adapter_config.json DELETED
@@ -1,50 +0,0 @@
1
- {
2
- "alora_invocation_tokens": null,
3
- "alpha_pattern": {},
4
- "arrow_config": null,
5
- "auto_mapping": {
6
- "base_model_class": "LlamaForCausalLM",
7
- "parent_library": "transformers.models.llama.modeling_llama",
8
- "unsloth_fixed": true
9
- },
10
- "base_model_name_or_path": "unsloth/Meta-Llama-3.1-8B",
11
- "bias": "none",
12
- "corda_config": null,
13
- "ensure_weight_tying": false,
14
- "eva_config": null,
15
- "exclude_modules": null,
16
- "fan_in_fan_out": false,
17
- "inference_mode": true,
18
- "init_lora_weights": true,
19
- "layer_replication": null,
20
- "layers_pattern": null,
21
- "layers_to_transform": null,
22
- "loftq_config": {},
23
- "lora_alpha": 16,
24
- "lora_bias": false,
25
- "lora_dropout": 0.0,
26
- "megatron_config": null,
27
- "megatron_core": "megatron.core",
28
- "modules_to_save": null,
29
- "peft_type": "LORA",
30
- "peft_version": "0.18.1",
31
- "qalora_group_size": 16,
32
- "r": 8,
33
- "rank_pattern": {},
34
- "revision": null,
35
- "target_modules": [
36
- "down_proj",
37
- "v_proj",
38
- "gate_proj",
39
- "up_proj",
40
- "o_proj",
41
- "q_proj",
42
- "k_proj"
43
- ],
44
- "target_parameters": null,
45
- "task_type": "CAUSAL_LM",
46
- "trainable_token_indices": null,
47
- "use_dora": false,
48
- "use_qalora": false,
49
- "use_rslora": false
50
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:1767ecc8a0b6be3fcc0530980c0b79b0669b0574b3e5eb9e24fe4f8f2da2d1dd
3
- size 83945296
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/chat_template.jinja DELETED
@@ -1,12 +0,0 @@
1
- {{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ messages[0]['content'] + '
2
-
3
- ' }}{% set loop_messages = messages[1:] %}{% else %}{{ '' + '
4
-
5
- ' }}{% set loop_messages = messages %}{% endif %}{% for message in loop_messages %}{% if message['role'] == 'user' %}{{ '### Instruction:
6
- ' + message['content'] + '
7
-
8
- ' }}{% elif message['role'] == 'assistant' %}{{ '### Response:
9
- ' + message['content'] + eos_token + '
10
-
11
- ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '### Response:
12
- ' }}{% endif %}
 
 
 
 
 
 
 
 
 
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:9bcb770bf5ce40ac20edec3ef11ea6e5346880a52a385b9e42ba26bbb415f744
3
- size 43127973
 
 
 
 
meta-llama__meta-llama-3.1-8b/sft/checkpoints/checkpoint-17592/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:419e357cd97f0d0fc2ead1cab9470f1570a1a1a10f851be6facc831c815ac5e0
3
- size 14645