Upload 12 files

lora_li_V5/checkpoint-350/README.md

@@ -0,0 +1,207 @@
+---
+library_name: peft
+base_model: line-corporation/japanese-large-lm-3.6b
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Data Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+## Training procedure
+
+
+### Framework versions
+
+
+- PEFT 0.6.2

lora_li_V5/checkpoint-350/adapter_config.json

@@ -0,0 +1,23 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "line-corporation/japanese-large-lm-3.6b",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 8,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 24,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "dense",
+    "query_key_value"
+  ],
+  "task_type": "CAUSAL_LM"
+}

lora_li_V5/checkpoint-350/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:293e8c7388b1d61f4c5c32375b64fe91c39c48fe5fa3af824f4d077776420d34
+size 53128554

lora_li_V5/checkpoint-350/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a3049363307409363c67a1ecccf6474acc2df025acac8fb4ca8ee6b358f1ef5
+size 26730618

lora_li_V5/checkpoint-350/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8581e8b68dc6c6196afee9e66ea321c280a0753f1cbf05a1d267b0cbec773f1e
+size 14244

lora_li_V5/checkpoint-350/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f9945ea8e74b658f882c54c4765f842ed7997c3e0788b118d20f344f68ad639
+size 1064

lora_li_V5/checkpoint-350/special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<cls>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "pad_token": "<pad>",
+  "sep_token": "<sep>",
+  "unk_token": "<unk>"
+}

lora_li_V5/checkpoint-350/spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c5c56a739832923347681ed8a03a9cbf5afb6d1fe60089a5b01dd2dd063ab71
+size 1208648

lora_li_V5/checkpoint-350/tokenizer_config.json

@@ -0,0 +1,18 @@
+{
+  "additional_special_tokens": null,
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<cls>",
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "extra_ids": 0,
+  "keep_accents": true,
+  "legacy": true,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sep_token": "<sep>",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "T5Tokenizer",
+  "unk_token": "<unk>"
+}

lora_li_V5/checkpoint-350/trainer_state.json

@@ -0,0 +1,999 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 7.0,
+  "eval_steps": 5,
+  "global_step": 350,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 3.7048,
+      "step": 5
+    },
+    {
+      "epoch": 0.1,
+      "eval_loss": 3.791994571685791,
+      "eval_runtime": 4.4098,
+      "eval_samples_per_second": 45.354,
+      "eval_steps_per_second": 5.669,
+      "step": 5
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 3.3333333333333335e-05,
+      "loss": 3.8064,
+      "step": 10
+    },
+    {
+      "epoch": 0.2,
+      "eval_loss": 3.777104377746582,
+      "eval_runtime": 4.422,
+      "eval_samples_per_second": 45.229,
+      "eval_steps_per_second": 5.654,
+      "step": 10
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 5e-05,
+      "loss": 3.7275,
+      "step": 15
+    },
+    {
+      "epoch": 0.3,
+      "eval_loss": 3.733438014984131,
+      "eval_runtime": 4.4275,
+      "eval_samples_per_second": 45.172,
+      "eval_steps_per_second": 5.646,
+      "step": 15
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 3.7422,
+      "step": 20
+    },
+    {
+      "epoch": 0.4,
+      "eval_loss": 3.6596221923828125,
+      "eval_runtime": 4.4331,
+      "eval_samples_per_second": 45.115,
+      "eval_steps_per_second": 5.639,
+      "step": 20
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 8.333333333333334e-05,
+      "loss": 3.637,
+      "step": 25
+    },
+    {
+      "epoch": 0.5,
+      "eval_loss": 3.5282764434814453,
+      "eval_runtime": 4.4412,
+      "eval_samples_per_second": 45.033,
+      "eval_steps_per_second": 5.629,
+      "step": 25
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 0.0001,
+      "loss": 3.4441,
+      "step": 30
+    },
+    {
+      "epoch": 0.6,
+      "eval_loss": 3.320693254470825,
+      "eval_runtime": 4.4455,
+      "eval_samples_per_second": 44.99,
+      "eval_steps_per_second": 5.624,
+      "step": 30
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 9.948453608247423e-05,
+      "loss": 3.2032,
+      "step": 35
+    },
+    {
+      "epoch": 0.7,
+      "eval_loss": 3.0499627590179443,
+      "eval_runtime": 4.4488,
+      "eval_samples_per_second": 44.956,
+      "eval_steps_per_second": 5.619,
+      "step": 35
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 9.896907216494846e-05,
+      "loss": 2.8873,
+      "step": 40
+    },
+    {
+      "epoch": 0.8,
+      "eval_loss": 2.7090697288513184,
+      "eval_runtime": 4.4542,
+      "eval_samples_per_second": 44.902,
+      "eval_steps_per_second": 5.613,
+      "step": 40
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 9.845360824742269e-05,
+      "loss": 2.5119,
+      "step": 45
+    },
+    {
+      "epoch": 0.9,
+      "eval_loss": 2.2660412788391113,
+      "eval_runtime": 4.4547,
+      "eval_samples_per_second": 44.896,
+      "eval_steps_per_second": 5.612,
+      "step": 45
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 9.793814432989691e-05,
+      "loss": 1.8782,
+      "step": 50
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.9092340469360352,
+      "eval_runtime": 4.4568,
+      "eval_samples_per_second": 44.876,
+      "eval_steps_per_second": 5.609,
+      "step": 50
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 9.742268041237114e-05,
+      "loss": 1.9562,
+      "step": 55
+    },
+    {
+      "epoch": 1.1,
+      "eval_loss": 1.6684613227844238,
+      "eval_runtime": 4.458,
+      "eval_samples_per_second": 44.863,
+      "eval_steps_per_second": 5.608,
+      "step": 55
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 9.690721649484537e-05,
+      "loss": 1.4385,
+      "step": 60
+    },
+    {
+      "epoch": 1.2,
+      "eval_loss": 1.4644845724105835,
+      "eval_runtime": 4.4554,
+      "eval_samples_per_second": 44.889,
+      "eval_steps_per_second": 5.611,
+      "step": 60
+    },
+    {
+      "epoch": 1.3,
+      "learning_rate": 9.639175257731959e-05,
+      "loss": 1.4579,
+      "step": 65
+    },
+    {
+      "epoch": 1.3,
+      "eval_loss": 1.323061227798462,
+      "eval_runtime": 4.4568,
+      "eval_samples_per_second": 44.875,
+      "eval_steps_per_second": 5.609,
+      "step": 65
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 9.587628865979382e-05,
+      "loss": 1.2973,
+      "step": 70
+    },
+    {
+      "epoch": 1.4,
+      "eval_loss": 1.296040654182434,
+      "eval_runtime": 4.4572,
+      "eval_samples_per_second": 44.871,
+      "eval_steps_per_second": 5.609,
+      "step": 70
+    },
+    {
+      "epoch": 1.5,
+      "learning_rate": 9.536082474226805e-05,
+      "loss": 1.0638,
+      "step": 75
+    },
+    {
+      "epoch": 1.5,
+      "eval_loss": 1.2820910215377808,
+      "eval_runtime": 4.4578,
+      "eval_samples_per_second": 44.865,
+      "eval_steps_per_second": 5.608,
+      "step": 75
+    },
+    {
+      "epoch": 1.6,
+      "learning_rate": 9.484536082474227e-05,
+      "loss": 1.3853,
+      "step": 80
+    },
+    {
+      "epoch": 1.6,
+      "eval_loss": 1.273141622543335,
+      "eval_runtime": 4.4582,
+      "eval_samples_per_second": 44.861,
+      "eval_steps_per_second": 5.608,
+      "step": 80
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 9.43298969072165e-05,
+      "loss": 0.9734,
+      "step": 85
+    },
+    {
+      "epoch": 1.7,
+      "eval_loss": 1.2661323547363281,
+      "eval_runtime": 4.4571,
+      "eval_samples_per_second": 44.873,
+      "eval_steps_per_second": 5.609,
+      "step": 85
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 9.381443298969073e-05,
+      "loss": 1.4012,
+      "step": 90
+    },
+    {
+      "epoch": 1.8,
+      "eval_loss": 1.2609992027282715,
+      "eval_runtime": 4.4568,
+      "eval_samples_per_second": 44.875,
+      "eval_steps_per_second": 5.609,
+      "step": 90
+    },
+    {
+      "epoch": 1.9,
+      "learning_rate": 9.329896907216495e-05,
+      "loss": 1.1392,
+      "step": 95
+    },
+    {
+      "epoch": 1.9,
+      "eval_loss": 1.2571262121200562,
+      "eval_runtime": 4.4597,
+      "eval_samples_per_second": 44.846,
+      "eval_steps_per_second": 5.606,
+      "step": 95
+    },
+    {
+      "epoch": 2.0,
+      "learning_rate": 9.278350515463918e-05,
+      "loss": 0.9244,
+      "step": 100
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2540690898895264,
+      "eval_runtime": 4.458,
+      "eval_samples_per_second": 44.864,
+      "eval_steps_per_second": 5.608,
+      "step": 100
+    },
+    {
+      "epoch": 2.1,
+      "learning_rate": 9.22680412371134e-05,
+      "loss": 1.521,
+      "step": 105
+    },
+    {
+      "epoch": 2.1,
+      "eval_loss": 1.2517199516296387,
+      "eval_runtime": 4.4571,
+      "eval_samples_per_second": 44.872,
+      "eval_steps_per_second": 5.609,
+      "step": 105
+    },
+    {
+      "epoch": 2.2,
+      "learning_rate": 9.175257731958763e-05,
+      "loss": 1.0514,
+      "step": 110
+    },
+    {
+      "epoch": 2.2,
+      "eval_loss": 1.2495222091674805,
+      "eval_runtime": 4.4586,
+      "eval_samples_per_second": 44.857,
+      "eval_steps_per_second": 5.607,
+      "step": 110
+    },
+    {
+      "epoch": 2.3,
+      "learning_rate": 9.123711340206186e-05,
+      "loss": 1.2462,
+      "step": 115
+    },
+    {
+      "epoch": 2.3,
+      "eval_loss": 1.2475061416625977,
+      "eval_runtime": 4.4596,
+      "eval_samples_per_second": 44.848,
+      "eval_steps_per_second": 5.606,
+      "step": 115
+    },
+    {
+      "epoch": 2.4,
+      "learning_rate": 9.072164948453609e-05,
+      "loss": 1.1681,
+      "step": 120
+    },
+    {
+      "epoch": 2.4,
+      "eval_loss": 1.2455750703811646,
+      "eval_runtime": 4.46,
+      "eval_samples_per_second": 44.843,
+      "eval_steps_per_second": 5.605,
+      "step": 120
+    },
+    {
+      "epoch": 2.5,
+      "learning_rate": 9.020618556701031e-05,
+      "loss": 1.0076,
+      "step": 125
+    },
+    {
+      "epoch": 2.5,
+      "eval_loss": 1.2440249919891357,
+      "eval_runtime": 4.4586,
+      "eval_samples_per_second": 44.857,
+      "eval_steps_per_second": 5.607,
+      "step": 125
+    },
+    {
+      "epoch": 2.6,
+      "learning_rate": 8.969072164948454e-05,
+      "loss": 1.4385,
+      "step": 130
+    },
+    {
+      "epoch": 2.6,
+      "eval_loss": 1.2428377866744995,
+      "eval_runtime": 4.4618,
+      "eval_samples_per_second": 44.825,
+      "eval_steps_per_second": 5.603,
+      "step": 130
+    },
+    {
+      "epoch": 2.7,
+      "learning_rate": 8.917525773195877e-05,
+      "loss": 0.9315,
+      "step": 135
+    },
+    {
+      "epoch": 2.7,
+      "eval_loss": 1.241697907447815,
+      "eval_runtime": 4.4589,
+      "eval_samples_per_second": 44.855,
+      "eval_steps_per_second": 5.607,
+      "step": 135
+    },
+    {
+      "epoch": 2.8,
+      "learning_rate": 8.865979381443299e-05,
+      "loss": 1.3113,
+      "step": 140
+    },
+    {
+      "epoch": 2.8,
+      "eval_loss": 1.240850567817688,
+      "eval_runtime": 4.4585,
+      "eval_samples_per_second": 44.859,
+      "eval_steps_per_second": 5.607,
+      "step": 140
+    },
+    {
+      "epoch": 2.9,
+      "learning_rate": 8.814432989690722e-05,
+      "loss": 1.1529,
+      "step": 145
+    },
+    {
+      "epoch": 2.9,
+      "eval_loss": 1.2396690845489502,
+      "eval_runtime": 4.4584,
+      "eval_samples_per_second": 44.859,
+      "eval_steps_per_second": 5.607,
+      "step": 145
+    },
+    {
+      "epoch": 3.0,
+      "learning_rate": 8.762886597938145e-05,
+      "loss": 0.919,
+      "step": 150
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.238713264465332,
+      "eval_runtime": 4.4607,
+      "eval_samples_per_second": 44.836,
+      "eval_steps_per_second": 5.604,
+      "step": 150
+    },
+    {
+      "epoch": 3.1,
+      "learning_rate": 8.711340206185567e-05,
+      "loss": 1.4306,
+      "step": 155
+    },
+    {
+      "epoch": 3.1,
+      "eval_loss": 1.2377328872680664,
+      "eval_runtime": 4.4589,
+      "eval_samples_per_second": 44.854,
+      "eval_steps_per_second": 5.607,
+      "step": 155
+    },
+    {
+      "epoch": 3.2,
+      "learning_rate": 8.65979381443299e-05,
+      "loss": 0.9846,
+      "step": 160
+    },
+    {
+      "epoch": 3.2,
+      "eval_loss": 1.2368167638778687,
+      "eval_runtime": 4.4586,
+      "eval_samples_per_second": 44.857,
+      "eval_steps_per_second": 5.607,
+      "step": 160
+    },
+    {
+      "epoch": 3.3,
+      "learning_rate": 8.608247422680413e-05,
+      "loss": 1.2453,
+      "step": 165
+    },
+    {
+      "epoch": 3.3,
+      "eval_loss": 1.2363089323043823,
+      "eval_runtime": 4.4577,
+      "eval_samples_per_second": 44.866,
+      "eval_steps_per_second": 5.608,
+      "step": 165
+    },
+    {
+      "epoch": 3.4,
+      "learning_rate": 8.556701030927835e-05,
+      "loss": 1.1861,
+      "step": 170
+    },
+    {
+      "epoch": 3.4,
+      "eval_loss": 1.235287070274353,
+      "eval_runtime": 4.458,
+      "eval_samples_per_second": 44.863,
+      "eval_steps_per_second": 5.608,
+      "step": 170
+    },
+    {
+      "epoch": 3.5,
+      "learning_rate": 8.505154639175259e-05,
+      "loss": 0.9753,
+      "step": 175
+    },
+    {
+      "epoch": 3.5,
+      "eval_loss": 1.2343323230743408,
+      "eval_runtime": 4.5175,
+      "eval_samples_per_second": 44.272,
+      "eval_steps_per_second": 5.534,
+      "step": 175
+    },
+    {
+      "epoch": 3.6,
+      "learning_rate": 8.453608247422681e-05,
+      "loss": 1.3656,
+      "step": 180
+    },
+    {
+      "epoch": 3.6,
+      "eval_loss": 1.2331607341766357,
+      "eval_runtime": 4.4647,
+      "eval_samples_per_second": 44.796,
+      "eval_steps_per_second": 5.599,
+      "step": 180
+    },
+    {
+      "epoch": 3.7,
+      "learning_rate": 8.402061855670103e-05,
+      "loss": 0.9298,
+      "step": 185
+    },
+    {
+      "epoch": 3.7,
+      "eval_loss": 1.2324612140655518,
+      "eval_runtime": 4.4692,
+      "eval_samples_per_second": 44.751,
+      "eval_steps_per_second": 5.594,
+      "step": 185
+    },
+    {
+      "epoch": 3.8,
+      "learning_rate": 8.350515463917527e-05,
+      "loss": 1.3496,
+      "step": 190
+    },
+    {
+      "epoch": 3.8,
+      "eval_loss": 1.2315438985824585,
+      "eval_runtime": 4.4629,
+      "eval_samples_per_second": 44.814,
+      "eval_steps_per_second": 5.602,
+      "step": 190
+    },
+    {
+      "epoch": 3.9,
+      "learning_rate": 8.298969072164949e-05,
+      "loss": 1.1524,
+      "step": 195
+    },
+    {
+      "epoch": 3.9,
+      "eval_loss": 1.2303318977355957,
+      "eval_runtime": 4.466,
+      "eval_samples_per_second": 44.783,
+      "eval_steps_per_second": 5.598,
+      "step": 195
+    },
+    {
+      "epoch": 4.0,
+      "learning_rate": 8.247422680412371e-05,
+      "loss": 0.9595,
+      "step": 200
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.2294974327087402,
+      "eval_runtime": 4.4618,
+      "eval_samples_per_second": 44.825,
+      "eval_steps_per_second": 5.603,
+      "step": 200
+    },
+    {
+      "epoch": 4.1,
+      "learning_rate": 8.195876288659795e-05,
+      "loss": 1.4978,
+      "step": 205
+    },
+    {
+      "epoch": 4.1,
+      "eval_loss": 1.228881597518921,
+      "eval_runtime": 4.4634,
+      "eval_samples_per_second": 44.809,
+      "eval_steps_per_second": 5.601,
+      "step": 205
+    },
+    {
+      "epoch": 4.2,
+      "learning_rate": 8.144329896907217e-05,
+      "loss": 1.0768,
+      "step": 210
+    },
+    {
+      "epoch": 4.2,
+      "eval_loss": 1.2280452251434326,
+      "eval_runtime": 4.4676,
+      "eval_samples_per_second": 44.766,
+      "eval_steps_per_second": 5.596,
+      "step": 210
+    },
+    {
+      "epoch": 4.3,
+      "learning_rate": 8.092783505154639e-05,
+      "loss": 1.2207,
+      "step": 215
+    },
+    {
+      "epoch": 4.3,
+      "eval_loss": 1.2275255918502808,
+      "eval_runtime": 4.4621,
+      "eval_samples_per_second": 44.822,
+      "eval_steps_per_second": 5.603,
+      "step": 215
+    },
+    {
+      "epoch": 4.4,
+      "learning_rate": 8.041237113402063e-05,
+      "loss": 1.0792,
+      "step": 220
+    },
+    {
+      "epoch": 4.4,
+      "eval_loss": 1.2285792827606201,
+      "eval_runtime": 4.4624,
+      "eval_samples_per_second": 44.819,
+      "eval_steps_per_second": 5.602,
+      "step": 220
+    },
+    {
+      "epoch": 4.5,
+      "learning_rate": 7.989690721649485e-05,
+      "loss": 0.9888,
+      "step": 225
+    },
+    {
+      "epoch": 4.5,
+      "eval_loss": 1.227547287940979,
+      "eval_runtime": 4.4611,
+      "eval_samples_per_second": 44.832,
+      "eval_steps_per_second": 5.604,
+      "step": 225
+    },
+    {
+      "epoch": 4.6,
+      "learning_rate": 7.938144329896907e-05,
+      "loss": 1.4156,
+      "step": 230
+    },
+    {
+      "epoch": 4.6,
+      "eval_loss": 1.2268065214157104,
+      "eval_runtime": 4.4631,
+      "eval_samples_per_second": 44.811,
+      "eval_steps_per_second": 5.601,
+      "step": 230
+    },
+    {
+      "epoch": 4.7,
+      "learning_rate": 7.88659793814433e-05,
+      "loss": 0.8983,
+      "step": 235
+    },
+    {
+      "epoch": 4.7,
+      "eval_loss": 1.2261872291564941,
+      "eval_runtime": 4.4627,
+      "eval_samples_per_second": 44.816,
+      "eval_steps_per_second": 5.602,
+      "step": 235
+    },
+    {
+      "epoch": 4.8,
+      "learning_rate": 7.835051546391753e-05,
+      "loss": 1.2464,
+      "step": 240
+    },
+    {
+      "epoch": 4.8,
+      "eval_loss": 1.2259939908981323,
+      "eval_runtime": 4.4625,
+      "eval_samples_per_second": 44.818,
+      "eval_steps_per_second": 5.602,
+      "step": 240
+    },
+    {
+      "epoch": 4.9,
+      "learning_rate": 7.783505154639175e-05,
+      "loss": 1.0682,
+      "step": 245
+    },
+    {
+      "epoch": 4.9,
+      "eval_loss": 1.2249301671981812,
+      "eval_runtime": 4.4625,
+      "eval_samples_per_second": 44.818,
+      "eval_steps_per_second": 5.602,
+      "step": 245
+    },
+    {
+      "epoch": 5.0,
+      "learning_rate": 7.731958762886599e-05,
+      "loss": 0.9483,
+      "step": 250
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.224156141281128,
+      "eval_runtime": 4.4608,
+      "eval_samples_per_second": 44.835,
+      "eval_steps_per_second": 5.604,
+      "step": 250
+    },
+    {
+      "epoch": 5.1,
+      "learning_rate": 7.680412371134021e-05,
+      "loss": 1.4381,
+      "step": 255
+    },
+    {
+      "epoch": 5.1,
+      "eval_loss": 1.2236390113830566,
+      "eval_runtime": 4.4621,
+      "eval_samples_per_second": 44.822,
+      "eval_steps_per_second": 5.603,
+      "step": 255
+    },
+    {
+      "epoch": 5.2,
+      "learning_rate": 7.628865979381443e-05,
+      "loss": 0.948,
+      "step": 260
+    },
+    {
+      "epoch": 5.2,
+      "eval_loss": 1.2235738039016724,
+      "eval_runtime": 4.4614,
+      "eval_samples_per_second": 44.829,
+      "eval_steps_per_second": 5.604,
+      "step": 260
+    },
+    {
+      "epoch": 5.3,
+      "learning_rate": 7.577319587628867e-05,
+      "loss": 1.2122,
+      "step": 265
+    },
+    {
+      "epoch": 5.3,
+      "eval_loss": 1.2235658168792725,
+      "eval_runtime": 4.4625,
+      "eval_samples_per_second": 44.818,
+      "eval_steps_per_second": 5.602,
+      "step": 265
+    },
+    {
+      "epoch": 5.4,
+      "learning_rate": 7.525773195876289e-05,
+      "loss": 1.1774,
+      "step": 270
+    },
+    {
+      "epoch": 5.4,
+      "eval_loss": 1.223145842552185,
+      "eval_runtime": 4.4677,
+      "eval_samples_per_second": 44.766,
+      "eval_steps_per_second": 5.596,
+      "step": 270
+    },
+    {
+      "epoch": 5.5,
+      "learning_rate": 7.474226804123711e-05,
+      "loss": 1.0283,
+      "step": 275
+    },
+    {
+      "epoch": 5.5,
+      "eval_loss": 1.2216259241104126,
+      "eval_runtime": 4.4922,
+      "eval_samples_per_second": 44.522,
+      "eval_steps_per_second": 5.565,
+      "step": 275
+    },
+    {
+      "epoch": 5.6,
+      "learning_rate": 7.422680412371135e-05,
+      "loss": 1.3049,
+      "step": 280
+    },
+    {
+      "epoch": 5.6,
+      "eval_loss": 1.2210520505905151,
+      "eval_runtime": 4.4641,
+      "eval_samples_per_second": 44.801,
+      "eval_steps_per_second": 5.6,
+      "step": 280
+    },
+    {
+      "epoch": 5.7,
+      "learning_rate": 7.371134020618557e-05,
+      "loss": 0.9682,
+      "step": 285
+    },
+    {
+      "epoch": 5.7,
+      "eval_loss": 1.221419095993042,
+      "eval_runtime": 4.4577,
+      "eval_samples_per_second": 44.866,
+      "eval_steps_per_second": 5.608,
+      "step": 285
+    },
+    {
+      "epoch": 5.8,
+      "learning_rate": 7.319587628865979e-05,
+      "loss": 1.2808,
+      "step": 290
+    },
+    {
+      "epoch": 5.8,
+      "eval_loss": 1.2211226224899292,
+      "eval_runtime": 4.475,
+      "eval_samples_per_second": 44.693,
+      "eval_steps_per_second": 5.587,
+      "step": 290
+    },
+    {
+      "epoch": 5.9,
+      "learning_rate": 7.268041237113403e-05,
+      "loss": 1.0147,
+      "step": 295
+    },
+    {
+      "epoch": 5.9,
+      "eval_loss": 1.2203408479690552,
+      "eval_runtime": 4.4586,
+      "eval_samples_per_second": 44.857,
+      "eval_steps_per_second": 5.607,
+      "step": 295
+    },
+    {
+      "epoch": 6.0,
+      "learning_rate": 7.216494845360825e-05,
+      "loss": 0.9456,
+      "step": 300
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.220377802848816,
+      "eval_runtime": 4.4629,
+      "eval_samples_per_second": 44.814,
+      "eval_steps_per_second": 5.602,
+      "step": 300
+    },
+    {
+      "epoch": 6.1,
+      "learning_rate": 7.164948453608247e-05,
+      "loss": 1.4398,
+      "step": 305
+    },
+    {
+      "epoch": 6.1,
+      "eval_loss": 1.2200127840042114,
+      "eval_runtime": 4.505,
+      "eval_samples_per_second": 44.396,
+      "eval_steps_per_second": 5.549,
+      "step": 305
+    },
+    {
+      "epoch": 6.2,
+      "learning_rate": 7.113402061855671e-05,
+      "loss": 0.9325,
+      "step": 310
+    },
+    {
+      "epoch": 6.2,
+      "eval_loss": 1.2199076414108276,
+      "eval_runtime": 4.4582,
+      "eval_samples_per_second": 44.861,
+      "eval_steps_per_second": 5.608,
+      "step": 310
+    },
+    {
+      "epoch": 6.3,
+      "learning_rate": 7.061855670103093e-05,
+      "loss": 1.2216,
+      "step": 315
+    },
+    {
+      "epoch": 6.3,
+      "eval_loss": 1.2199695110321045,
+      "eval_runtime": 4.4598,
+      "eval_samples_per_second": 44.845,
+      "eval_steps_per_second": 5.606,
+      "step": 315
+    },
+    {
+      "epoch": 6.4,
+      "learning_rate": 7.010309278350515e-05,
+      "loss": 1.1325,
+      "step": 320
+    },
+    {
+      "epoch": 6.4,
+      "eval_loss": 1.2204923629760742,
+      "eval_runtime": 4.4636,
+      "eval_samples_per_second": 44.807,
+      "eval_steps_per_second": 5.601,
+      "step": 320
+    },
+    {
+      "epoch": 6.5,
+      "learning_rate": 6.958762886597939e-05,
+      "loss": 0.9404,
+      "step": 325
+    },
+    {
+      "epoch": 6.5,
+      "eval_loss": 1.2200615406036377,
+      "eval_runtime": 4.4808,
+      "eval_samples_per_second": 44.635,
+      "eval_steps_per_second": 5.579,
+      "step": 325
+    },
+    {
+      "epoch": 6.6,
+      "learning_rate": 6.907216494845361e-05,
+      "loss": 1.3678,
+      "step": 330
+    },
+    {
+      "epoch": 6.6,
+      "eval_loss": 1.2192314863204956,
+      "eval_runtime": 4.4613,
+      "eval_samples_per_second": 44.83,
+      "eval_steps_per_second": 5.604,
+      "step": 330
+    },
+    {
+      "epoch": 6.7,
+      "learning_rate": 6.855670103092783e-05,
+      "loss": 0.9207,
+      "step": 335
+    },
+    {
+      "epoch": 6.7,
+      "eval_loss": 1.218888759613037,
+      "eval_runtime": 4.462,
+      "eval_samples_per_second": 44.823,
+      "eval_steps_per_second": 5.603,
+      "step": 335
+    },
+    {
+      "epoch": 6.8,
+      "learning_rate": 6.804123711340207e-05,
+      "loss": 1.2759,
+      "step": 340
+    },
+    {
+      "epoch": 6.8,
+      "eval_loss": 1.2193782329559326,
+      "eval_runtime": 4.4681,
+      "eval_samples_per_second": 44.762,
+      "eval_steps_per_second": 5.595,
+      "step": 340
+    },
+    {
+      "epoch": 6.9,
+      "learning_rate": 6.752577319587629e-05,
+      "loss": 1.0964,
+      "step": 345
+    },
+    {
+      "epoch": 6.9,
+      "eval_loss": 1.2190594673156738,
+      "eval_runtime": 4.4642,
+      "eval_samples_per_second": 44.8,
+      "eval_steps_per_second": 5.6,
+      "step": 345
+    },
+    {
+      "epoch": 7.0,
+      "learning_rate": 6.701030927835051e-05,
+      "loss": 0.8757,
+      "step": 350
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 1.2184209823608398,
+      "eval_runtime": 4.4666,
+      "eval_samples_per_second": 44.777,
+      "eval_steps_per_second": 5.597,
+      "step": 350
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1000,
+  "num_train_epochs": 20,
+  "save_steps": 10,
+  "total_flos": 1.2443768572280832e+16,
+  "trial_name": null,
+  "trial_params": null
+}

lora_li_V5/checkpoint-350/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c6559144927c60494ec69fd4b328d5b34050e0192f6d1aa6f089d59b66616b2
+size 4472