Instructions to use LayerEight/Wikipedia-Networking-Qwen2.5-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LayerEight/Wikipedia-Networking-Qwen2.5-7B with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base_model, "LayerEight/Wikipedia-Networking-Qwen2.5-7B")

Transformers

How to use LayerEight/Wikipedia-Networking-Qwen2.5-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LayerEight/Wikipedia-Networking-Qwen2.5-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("LayerEight/Wikipedia-Networking-Qwen2.5-7B", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use LayerEight/Wikipedia-Networking-Qwen2.5-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LayerEight/Wikipedia-Networking-Qwen2.5-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LayerEight/Wikipedia-Networking-Qwen2.5-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LayerEight/Wikipedia-Networking-Qwen2.5-7B

SGLang

How to use LayerEight/Wikipedia-Networking-Qwen2.5-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LayerEight/Wikipedia-Networking-Qwen2.5-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LayerEight/Wikipedia-Networking-Qwen2.5-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LayerEight/Wikipedia-Networking-Qwen2.5-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LayerEight/Wikipedia-Networking-Qwen2.5-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LayerEight/Wikipedia-Networking-Qwen2.5-7B with Docker Model Runner:
```
docker model run hf.co/LayerEight/Wikipedia-Networking-Qwen2.5-7B
```

LayerEight commited on 5 days ago

Commit

3fe1d1c

verified ·

1 Parent(s): 388483c

Upload Wikipedia-Networking-Qwen2.5-7B from Wikipedia-Networking-7B-output

Browse files

Files changed (30) hide show

.gitattributes +3 -0
README.md +62 -0
adapter_config.json +48 -0
adapter_model.safetensors +3 -0
chat_template.jinja +54 -0
checkpoint-1500/README.md +209 -0
checkpoint-1500/adapter_config.json +48 -0
checkpoint-1500/adapter_model.safetensors +3 -0
checkpoint-1500/chat_template.jinja +54 -0
checkpoint-1500/optimizer.pt +3 -0
checkpoint-1500/rng_state.pth +3 -0
checkpoint-1500/scheduler.pt +3 -0
checkpoint-1500/tokenizer.json +3 -0
checkpoint-1500/tokenizer_config.json +29 -0
checkpoint-1500/trainer_state.json +634 -0
checkpoint-1500/training_args.bin +3 -0
checkpoint-1788/README.md +209 -0
checkpoint-1788/adapter_config.json +48 -0
checkpoint-1788/adapter_model.safetensors +3 -0
checkpoint-1788/chat_template.jinja +54 -0
checkpoint-1788/optimizer.pt +3 -0
checkpoint-1788/rng_state.pth +3 -0
checkpoint-1788/scheduler.pt +3 -0
checkpoint-1788/tokenizer.json +3 -0
checkpoint-1788/tokenizer_config.json +29 -0
checkpoint-1788/trainer_state.json +744 -0
checkpoint-1788/training_args.bin +3 -0
tokenizer.json +3 -0
tokenizer_config.json +29 -0
training_args.bin +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-1500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1788/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,62 @@

+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+model_name: Wikipedia-Networking-7B-output
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+licence: license
+pipeline_tag: text-generation
+---
+# Model Card for Wikipedia-Networking-7B-output
+This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+## Quick start
+```python
+from transformers import pipeline
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+## Training procedure
+This model was trained with SFT.
+### Framework versions
+- PEFT 0.19.1
+- TRL: 1.2.0
+- Transformers: 5.5.1
+- Pytorch: 2.11.0+rocm7.2
+- Datasets: 4.8.4
+- Tokenizers: 0.22.2
+## Citations
+Cite TRL as:
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

adapter_config.json ADDED Viewed

	@@ -0,0 +1,48 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "k_proj",
+    "up_proj",
+    "gate_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:71ce8f09c288cc8c8b7ac6193695f7f42b8be0ba400b084b9163a8060c8df7bb
+size 80792880

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,54 @@

+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-1500/README.md ADDED Viewed

	@@ -0,0 +1,209 @@

+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+# Model Card for Model ID
+<!-- Provide a quick summary of what the model is/does. -->
+## Model Details
+### Model Description
+<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+### Model Sources [optional]
+<!-- Provide the basic links for the model. -->
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+## Uses
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+### Direct Use
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+[More Information Needed]
+### Downstream Use [optional]
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+[More Information Needed]
+### Out-of-Scope Use
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+[More Information Needed]
+## Bias, Risks, and Limitations
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+[More Information Needed]
+### Recommendations
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+## How to Get Started with the Model
+Use the code below to get started with the model.
+[More Information Needed]
+## Training Details
+### Training Data
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+[More Information Needed]
+### Training Procedure
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+#### Preprocessing [optional]
+[More Information Needed]
+#### Training Hyperparameters
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Speeds, Sizes, Times [optional]
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+[More Information Needed]
+## Evaluation
+<!-- This section describes the evaluation protocols and provides the results. -->
+### Testing Data, Factors & Metrics
+#### Testing Data
+<!-- This should link to a Dataset Card if possible. -->
+[More Information Needed]
+#### Factors
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+[More Information Needed]
+#### Metrics
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+[More Information Needed]
+### Results
+[More Information Needed]
+#### Summary
+## Model Examination [optional]
+<!-- Relevant interpretability work for the model goes here -->
+[More Information Needed]
+## Environmental Impact
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+## Technical Specifications [optional]
+### Model Architecture and Objective
+[More Information Needed]
+### Compute Infrastructure
+[More Information Needed]
+#### Hardware
+[More Information Needed]
+#### Software
+[More Information Needed]
+## Citation [optional]
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**BibTeX:**
+[More Information Needed]
+**APA:**
+[More Information Needed]
+## Glossary [optional]
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+[More Information Needed]
+## More Information [optional]
+[More Information Needed]
+## Model Card Authors [optional]
+[More Information Needed]
+## Model Card Contact
+[More Information Needed]
+### Framework versions
+- PEFT 0.19.1

checkpoint-1500/adapter_config.json ADDED Viewed

	@@ -0,0 +1,48 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "k_proj",
+    "up_proj",
+    "gate_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1500/adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d28fc138d3dffebe113f194603ad1661df4d1647602418576531f98b087c7cc0
+size 80792880

checkpoint-1500/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,54 @@

+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-1500/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:67e3ebe9bce90372e1a800b5f658ef47b9f65684cc0ab5ed06755a76bf5b77b4
+size 161810747

checkpoint-1500/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d860ff33925c1b772004b2d307980bb90150429efe82bfdbc121257969ecd5aa
+size 14645

checkpoint-1500/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8fb94d7208a78f2b9f0eaad152a2d3bdf7f4dc3b4166cc8ccf715eabb1c1337f
+size 1465

checkpoint-1500/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

checkpoint-1500/tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,29 @@

+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-1500/trainer_state.json ADDED Viewed

	@@ -0,0 +1,634 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.8393662784597629,
+  "eval_steps": 500,
+  "global_step": 1500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6571732765436173,
+      "epoch": 0.013989437974329381,
+      "grad_norm": 0.0257568359375,
+      "learning_rate": 8.888888888888889e-05,
+      "loss": 1.9518862915039064,
+      "mean_token_accuracy": 0.5733682353049516,
+      "num_tokens": 294540.0,
+      "step": 25
+    },
+    {
+      "entropy": 1.7126372032612562,
+      "epoch": 0.027978875948658763,
+      "grad_norm": 0.01416015625,
+      "learning_rate": 0.0001814814814814815,
+      "loss": 1.6628182983398438,
+      "mean_token_accuracy": 0.6153687073290348,
+      "num_tokens": 593048.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.6262741280719637,
+      "epoch": 0.041968313922988144,
+      "grad_norm": 0.01806640625,
+      "learning_rate": 0.00019993435766282977,
+      "loss": 1.584449005126953,
+      "mean_token_accuracy": 0.6257701878249645,
+      "num_tokens": 891346.0,
+      "step": 75
+    },
+    {
+      "entropy": 1.58288999453187,
+      "epoch": 0.055957751897317526,
+      "grad_norm": 0.016357421875,
+      "learning_rate": 0.00019966783335941717,
+      "loss": 1.5507936096191406,
+      "mean_token_accuracy": 0.6367124810814857,
+      "num_tokens": 1184611.0,
+      "step": 100
+    },
+    {
+      "entropy": 1.5433334041386844,
+      "epoch": 0.0699471898716469,
+      "grad_norm": 0.0189208984375,
+      "learning_rate": 0.00019919687071661834,
+      "loss": 1.512821044921875,
+      "mean_token_accuracy": 0.6423260240256786,
+      "num_tokens": 1480125.0,
+      "step": 125
+    },
+    {
+      "entropy": 1.560064207613468,
+      "epoch": 0.08393662784597629,
+      "grad_norm": 0.017822265625,
+      "learning_rate": 0.00019852243577149215,
+      "loss": 1.520980224609375,
+      "mean_token_accuracy": 0.6398584572970867,
+      "num_tokens": 1770905.0,
+      "step": 150
+    },
+    {
+      "entropy": 1.4850163259357214,
+      "epoch": 0.09792606582030566,
+      "grad_norm": 0.0235595703125,
+      "learning_rate": 0.00019764591192282993,
+      "loss": 1.460587158203125,
+      "mean_token_accuracy": 0.6578983479738235,
+      "num_tokens": 2061126.0,
+      "step": 175
+    },
+    {
+      "entropy": 1.4848217015340923,
+      "epoch": 0.11191550379463505,
+      "grad_norm": 0.03662109375,
+      "learning_rate": 0.00019656909709353236,
+      "loss": 1.4420854187011718,
+      "mean_token_accuracy": 0.6592926564067603,
+      "num_tokens": 2352476.0,
+      "step": 200
+    },
+    {
+      "entropy": 1.4575298309139908,
+      "epoch": 0.12590494176896444,
+      "grad_norm": 0.0196533203125,
+      "learning_rate": 0.00019529420004271567,
+      "loss": 1.4135537719726563,
+      "mean_token_accuracy": 0.6653153513371944,
+      "num_tokens": 2647293.0,
+      "step": 225
+    },
+    {
+      "entropy": 1.440113589912653,
+      "epoch": 0.1398943797432938,
+      "grad_norm": 0.0238037109375,
+      "learning_rate": 0.00019382383583511206,
+      "loss": 1.416180877685547,
+      "mean_token_accuracy": 0.6659375876933337,
+      "num_tokens": 2941859.0,
+      "step": 250
+    },
+    {
+      "entropy": 1.4387904461845755,
+      "epoch": 0.1538838177176232,
+      "grad_norm": 0.0263671875,
+      "learning_rate": 0.00019216102047705746,
+      "loss": 1.38541015625,
+      "mean_token_accuracy": 0.6683934908360243,
+      "num_tokens": 3237763.0,
+      "step": 275
+    },
+    {
+      "entropy": 1.3664818671159447,
+      "epoch": 0.16787325569195258,
+      "grad_norm": 0.02001953125,
+      "learning_rate": 0.00019030916473006888,
+      "loss": 1.3284611511230469,
+      "mean_token_accuracy": 0.6843961907178163,
+      "num_tokens": 3541854.0,
+      "step": 300
+    },
+    {
+      "entropy": 1.3906436143629253,
+      "epoch": 0.18186269366628197,
+      "grad_norm": 0.0296630859375,
+      "learning_rate": 0.00018827206711470137,
+      "loss": 1.3411039733886718,
+      "mean_token_accuracy": 0.6773389867693186,
+      "num_tokens": 3832012.0,
+      "step": 325
+    },
+    {
+      "entropy": 1.3563544462341814,
+      "epoch": 0.19585213164061133,
+      "grad_norm": 0.0233154296875,
+      "learning_rate": 0.000186053906119035,
+      "loss": 1.3131784057617188,
+      "mean_token_accuracy": 0.686054406017065,
+      "num_tokens": 4129238.0,
+      "step": 350
+    },
+    {
+      "entropy": 1.3752087260596455,
+      "epoch": 0.20984156961494072,
+      "grad_norm": 0.0272216796875,
+      "learning_rate": 0.00018365923162777336,
+      "loss": 1.338323974609375,
+      "mean_token_accuracy": 0.6795472690463066,
+      "num_tokens": 4414939.0,
+      "step": 375
+    },
+    {
+      "entropy": 1.3106766785029322,
+      "epoch": 0.2238310075892701,
+      "grad_norm": 0.027099609375,
+      "learning_rate": 0.0001810929555895348,
+      "loss": 1.2676075744628905,
+      "mean_token_accuracy": 0.6990727451443672,
+      "num_tokens": 4711065.0,
+      "step": 400
+    },
+    {
+      "entropy": 1.2741564935538918,
+      "epoch": 0.2378204455635995,
+      "grad_norm": 0.0341796875,
+      "learning_rate": 0.00017836034194147922,
+      "loss": 1.231330337524414,
+      "mean_token_accuracy": 0.7052560625225306,
+      "num_tokens": 5006515.0,
+      "step": 425
+    },
+    {
+      "entropy": 1.255356255499646,
+      "epoch": 0.2518098835379289,
+      "grad_norm": 0.025390625,
+      "learning_rate": 0.000175466995811937,
+      "loss": 1.2256485748291015,
+      "mean_token_accuracy": 0.709921350106597,
+      "num_tokens": 5298743.0,
+      "step": 450
+    },
+    {
+      "entropy": 1.2404796612542122,
+      "epoch": 0.26579932151225827,
+      "grad_norm": 0.029296875,
+      "learning_rate": 0.00017241885202318787,
+      "loss": 1.199592056274414,
+      "mean_token_accuracy": 0.7124969442933797,
+      "num_tokens": 5587292.0,
+      "step": 475
+    },
+    {
+      "entropy": 1.2135492172930389,
+      "epoch": 0.2797887594865876,
+      "grad_norm": 0.0341796875,
+      "learning_rate": 0.00016922216291797248,
+      "loss": 1.1718869781494141,
+      "mean_token_accuracy": 0.7194690197706223,
+      "num_tokens": 5878179.0,
+      "step": 500
+    },
+    {
+      "entropy": 1.1519292463734745,
+      "epoch": 0.293778197460917,
+      "grad_norm": 0.044921875,
+      "learning_rate": 0.0001658834855347071,
+      "loss": 1.1038724517822265,
+      "mean_token_accuracy": 0.7360028678923846,
+      "num_tokens": 6174073.0,
+      "step": 525
+    },
+    {
+      "entropy": 1.2023496214020997,
+      "epoch": 0.3077676354352464,
+      "grad_norm": 0.035400390625,
+      "learning_rate": 0.00016240966815770754,
+      "loss": 1.1433139801025392,
+      "mean_token_accuracy": 0.7237949234992266,
+      "num_tokens": 6471377.0,
+      "step": 550
+    },
+    {
+      "entropy": 1.2287814150052145,
+      "epoch": 0.32175707340957577,
+      "grad_norm": 0.0262451171875,
+      "learning_rate": 0.00015880783627001026,
+      "loss": 1.196635513305664,
+      "mean_token_accuracy": 0.7167265053838492,
+      "num_tokens": 6762946.0,
+      "step": 575
+    },
+    {
+      "entropy": 1.1513058760762214,
+      "epoch": 0.33574651138390516,
+      "grad_norm": 0.03759765625,
+      "learning_rate": 0.0001550853779376045,
+      "loss": 1.104804229736328,
+      "mean_token_accuracy": 0.7350956991314888,
+      "num_tokens": 7058166.0,
+      "step": 600
+    },
+    {
+      "entropy": 1.1233506935788318,
+      "epoch": 0.34973594935823454,
+      "grad_norm": 0.038818359375,
+      "learning_rate": 0.00015124992865505523,
+      "loss": 1.0836825561523438,
+      "mean_token_accuracy": 0.7427010469138622,
+      "num_tokens": 7351236.0,
+      "step": 625
+    },
+    {
+      "entropy": 1.1373229866661132,
+      "epoch": 0.36372538733256393,
+      "grad_norm": 0.03466796875,
+      "learning_rate": 0.00014730935568360102,
+      "loss": 1.0919375610351563,
+      "mean_token_accuracy": 0.7399685283005237,
+      "num_tokens": 7642275.0,
+      "step": 650
+    },
+    {
+      "entropy": 1.1448556556599214,
+      "epoch": 0.3777148253068933,
+      "grad_norm": 0.033447265625,
+      "learning_rate": 0.0001432717419138532,
+      "loss": 1.0814331817626952,
+      "mean_token_accuracy": 0.7378807676583529,
+      "num_tokens": 7935222.0,
+      "step": 675
+    },
+    {
+      "entropy": 1.040786258596927,
+      "epoch": 0.39170426328122265,
+      "grad_norm": 0.0311279296875,
+      "learning_rate": 0.0001391453692861967,
+      "loss": 0.9921630096435546,
+      "mean_token_accuracy": 0.7638325883448124,
+      "num_tokens": 8234237.0,
+      "step": 700
+    },
+    {
+      "entropy": 1.1662991707585753,
+      "epoch": 0.40569370125555204,
+      "grad_norm": 0.0302734375,
+      "learning_rate": 0.0001349387018029003,
+      "loss": 1.118931884765625,
+      "mean_token_accuracy": 0.732994829416275,
+      "num_tokens": 8529663.0,
+      "step": 725
+    },
+    {
+      "entropy": 1.049061678093858,
+      "epoch": 0.41968313922988143,
+      "grad_norm": 0.041015625,
+      "learning_rate": 0.00013066036816678273,
+      "loss": 0.9903451538085938,
+      "mean_token_accuracy": 0.7613946237415076,
+      "num_tokens": 8817967.0,
+      "step": 750
+    },
+    {
+      "entropy": 1.1110075389547274,
+      "epoch": 0.4336725772042108,
+      "grad_norm": 0.0361328125,
+      "learning_rate": 0.0001263191440820448,
+      "loss": 1.0462547302246095,
+      "mean_token_accuracy": 0.7477244459837675,
+      "num_tokens": 9107363.0,
+      "step": 775
+    },
+    {
+      "entropy": 1.024681367701851,
+      "epoch": 0.4476620151785402,
+      "grad_norm": 0.039306640625,
+      "learning_rate": 0.00012192393425357353,
+      "loss": 0.9708639526367188,
+      "mean_token_accuracy": 0.7660154252499342,
+      "num_tokens": 9398385.0,
+      "step": 800
+    },
+    {
+      "entropy": 1.0914598937472328,
+      "epoch": 0.4616514531528696,
+      "grad_norm": 0.03662109375,
+      "learning_rate": 0.00011748375412163984,
+      "loss": 1.0285856628417969,
+      "mean_token_accuracy": 0.7533510192483663,
+      "num_tokens": 9692995.0,
+      "step": 825
+    },
+    {
+      "entropy": 1.070437773577869,
+      "epoch": 0.475640891127199,
+      "grad_norm": 0.0308837890625,
+      "learning_rate": 0.00011300771136945658,
+      "loss": 1.002013931274414,
+      "mean_token_accuracy": 0.7574464529007673,
+      "num_tokens": 9985540.0,
+      "step": 850
+    },
+    {
+      "entropy": 1.0500496013974772,
+      "epoch": 0.48963032910152837,
+      "grad_norm": 0.033935546875,
+      "learning_rate": 0.00010850498724152798,
+      "loss": 1.0061278533935547,
+      "mean_token_accuracy": 0.7601391483098269,
+      "num_tokens": 10280858.0,
+      "step": 875
+    },
+    {
+      "entropy": 0.9829169751703739,
+      "epoch": 0.5036197670758578,
+      "grad_norm": 0.035888671875,
+      "learning_rate": 0.00010398481771111037,
+      "loss": 0.9166652679443359,
+      "mean_token_accuracy": 0.7795024861395359,
+      "num_tokens": 10571194.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.9858249768940732,
+      "epoch": 0.5176092050501871,
+      "grad_norm": 0.038818359375,
+      "learning_rate": 9.94564745354137e-05,
+      "loss": 0.9250814819335937,
+      "mean_token_accuracy": 0.7783324559032917,
+      "num_tokens": 10861535.0,
+      "step": 925
+    },
+    {
+      "entropy": 0.9617049301974475,
+      "epoch": 0.5315986430245165,
+      "grad_norm": 0.0419921875,
+      "learning_rate": 9.492924623740271e-05,
+      "loss": 0.9093881988525391,
+      "mean_token_accuracy": 0.7818463468551635,
+      "num_tokens": 11157232.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.9786588902026415,
+      "epoch": 0.5455880809988459,
+      "grad_norm": 0.031494140625,
+      "learning_rate": 9.04124190532087e-05,
+      "loss": 0.9290876007080078,
+      "mean_token_accuracy": 0.7784507688879967,
+      "num_tokens": 11451817.0,
+      "step": 975
+    },
+    {
+      "entropy": 0.9941231621522456,
+      "epoch": 0.5595775189731752,
+      "grad_norm": 0.03662109375,
+      "learning_rate": 8.591525788423168e-05,
+      "loss": 0.9281739807128906,
+      "mean_token_accuracy": 0.7773911864310503,
+      "num_tokens": 11746337.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.9537221440812573,
+      "epoch": 0.5735669569475046,
+      "grad_norm": 0.0303955078125,
+      "learning_rate": 8.144698729300455e-05,
+      "loss": 0.8921097564697266,
+      "mean_token_accuracy": 0.7855567722022534,
+      "num_tokens": 12037412.0,
+      "step": 1025
+    },
+    {
+      "entropy": 0.9557448910363019,
+      "epoch": 0.587556394921834,
+      "grad_norm": 0.029052734375,
+      "learning_rate": 7.701677258179996e-05,
+      "loss": 0.9018702697753906,
+      "mean_token_accuracy": 0.78552112005651,
+      "num_tokens": 12330073.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.9831135825067758,
+      "epoch": 0.6015458328961634,
+      "grad_norm": 0.031005859375,
+      "learning_rate": 7.263370099279172e-05,
+      "loss": 0.9220442199707031,
+      "mean_token_accuracy": 0.7789029122143983,
+      "num_tokens": 12623027.0,
+      "step": 1075
+    },
+    {
+      "entropy": 0.8838035979354754,
+      "epoch": 0.6155352708704928,
+      "grad_norm": 0.040283203125,
+      "learning_rate": 6.83067630683331e-05,
+      "loss": 0.8227420043945313,
+      "mean_token_accuracy": 0.8038330339640379,
+      "num_tokens": 12916473.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.883046350381337,
+      "epoch": 0.6295247088448221,
+      "grad_norm": 0.037353515625,
+      "learning_rate": 6.404483420958494e-05,
+      "loss": 0.8318240356445312,
+      "mean_token_accuracy": 0.8026394218951464,
+      "num_tokens": 13210350.0,
+      "step": 1125
+    },
+    {
+      "entropy": 0.9277670627878979,
+      "epoch": 0.6435141468191515,
+      "grad_norm": 0.02783203125,
+      "learning_rate": 5.9856656471321636e-05,
+      "loss": 0.8780873870849609,
+      "mean_token_accuracy": 0.7957132039964199,
+      "num_tokens": 13501483.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.9232714768499136,
+      "epoch": 0.6575035847934809,
+      "grad_norm": 0.0255126953125,
+      "learning_rate": 5.575082063025619e-05,
+      "loss": 0.8550299835205079,
+      "mean_token_accuracy": 0.7936624947935342,
+      "num_tokens": 13797592.0,
+      "step": 1175
+    },
+    {
+      "entropy": 0.899801154313609,
+      "epoch": 0.6714930227678103,
+      "grad_norm": 0.03369140625,
+      "learning_rate": 5.173574856366683e-05,
+      "loss": 0.8404230499267578,
+      "mean_token_accuracy": 0.7989044986665249,
+      "num_tokens": 14085126.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.8731577204307541,
+      "epoch": 0.6854824607421397,
+      "grad_norm": 0.033203125,
+      "learning_rate": 4.7819675974469356e-05,
+      "loss": 0.8161344146728515,
+      "mean_token_accuracy": 0.8059349462389946,
+      "num_tokens": 14383100.0,
+      "step": 1225
+    },
+    {
+      "entropy": 0.9630948188621551,
+      "epoch": 0.6994718987164691,
+      "grad_norm": 0.033935546875,
+      "learning_rate": 4.401063549816984e-05,
+      "loss": 0.91357177734375,
+      "mean_token_accuracy": 0.7841827914863825,
+      "num_tokens": 14682819.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.9606740972073748,
+      "epoch": 0.7134613366907985,
+      "grad_norm": 0.02783203125,
+      "learning_rate": 4.031644022634831e-05,
+      "loss": 0.896214599609375,
+      "mean_token_accuracy": 0.7852722837030888,
+      "num_tokens": 14972956.0,
+      "step": 1275
+    },
+    {
+      "entropy": 0.9392211610358209,
+      "epoch": 0.7274507746651279,
+      "grad_norm": 0.033447265625,
+      "learning_rate": 3.674466768047078e-05,
+      "loss": 0.8826549530029297,
+      "mean_token_accuracy": 0.7912480696290731,
+      "num_tokens": 15271172.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.9369741760706529,
+      "epoch": 0.7414402126394573,
+      "grad_norm": 0.0228271484375,
+      "learning_rate": 3.330264426890114e-05,
+      "loss": 0.8716287994384766,
+      "mean_token_accuracy": 0.7906265539675951,
+      "num_tokens": 15567414.0,
+      "step": 1325
+    },
+    {
+      "entropy": 0.8475008888402954,
+      "epoch": 0.7554296506137866,
+      "grad_norm": 0.02734375,
+      "learning_rate": 2.9997430258996208e-05,
+      "loss": 0.7837786865234375,
+      "mean_token_accuracy": 0.8113828019797802,
+      "num_tokens": 15862493.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.8544112277310342,
+      "epoch": 0.7694190885881159,
+      "grad_norm": 0.0400390625,
+      "learning_rate": 2.6835805295107897e-05,
+      "loss": 0.7910999298095703,
+      "mean_token_accuracy": 0.8121292810887099,
+      "num_tokens": 16154298.0,
+      "step": 1375
+    },
+    {
+      "entropy": 0.8836554439552128,
+      "epoch": 0.7834085265624453,
+      "grad_norm": 0.03271484375,
+      "learning_rate": 2.382425449219854e-05,
+      "loss": 0.8322444152832031,
+      "mean_token_accuracy": 0.8058089179545641,
+      "num_tokens": 16450244.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.9750821837875993,
+      "epoch": 0.7973979645367747,
+      "grad_norm": 0.0279541015625,
+      "learning_rate": 2.0968955133593805e-05,
+      "loss": 0.9110565948486328,
+      "mean_token_accuracy": 0.7827954424917698,
+      "num_tokens": 16742734.0,
+      "step": 1425
+    },
+    {
+      "entropy": 0.8045650019776076,
+      "epoch": 0.8113874025111041,
+      "grad_norm": 0.037841796875,
+      "learning_rate": 1.8275764000159222e-05,
+      "loss": 0.7386186218261719,
+      "mean_token_accuracy": 0.8229204141348601,
+      "num_tokens": 17040960.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.9300361970160157,
+      "epoch": 0.8253768404854335,
+      "grad_norm": 0.0303955078125,
+      "learning_rate": 1.5750205356889937e-05,
+      "loss": 0.8793846130371094,
+      "mean_token_accuracy": 0.7934108532965183,
+      "num_tokens": 17340398.0,
+      "step": 1475
+    },
+    {
+      "entropy": 0.9426807796442881,
+      "epoch": 0.8393662784597629,
+      "grad_norm": 0.032958984375,
+      "learning_rate": 1.339745962155613e-05,
+      "loss": 0.8905538940429687,
+      "mean_token_accuracy": 0.7892987384647131,
+      "num_tokens": 17629025.0,
+      "step": 1500
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 1788,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.521588501358848e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1500/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b0ea67ee3810c6e10021bd35d99633fb742a8db898d131b220e180054e0dcd7
+size 5649

checkpoint-1788/README.md ADDED Viewed

	@@ -0,0 +1,209 @@

+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+# Model Card for Model ID
+<!-- Provide a quick summary of what the model is/does. -->
+## Model Details
+### Model Description
+<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+### Model Sources [optional]
+<!-- Provide the basic links for the model. -->
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+## Uses
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+### Direct Use
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+[More Information Needed]
+### Downstream Use [optional]
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+[More Information Needed]
+### Out-of-Scope Use
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+[More Information Needed]
+## Bias, Risks, and Limitations
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+[More Information Needed]
+### Recommendations
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+## How to Get Started with the Model
+Use the code below to get started with the model.
+[More Information Needed]
+## Training Details
+### Training Data
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+[More Information Needed]
+### Training Procedure
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+#### Preprocessing [optional]
+[More Information Needed]
+#### Training Hyperparameters
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Speeds, Sizes, Times [optional]
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+[More Information Needed]
+## Evaluation
+<!-- This section describes the evaluation protocols and provides the results. -->
+### Testing Data, Factors & Metrics
+#### Testing Data
+<!-- This should link to a Dataset Card if possible. -->
+[More Information Needed]
+#### Factors
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+[More Information Needed]
+#### Metrics
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+[More Information Needed]
+### Results
+[More Information Needed]
+#### Summary
+## Model Examination [optional]
+<!-- Relevant interpretability work for the model goes here -->
+[More Information Needed]
+## Environmental Impact
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+## Technical Specifications [optional]
+### Model Architecture and Objective
+[More Information Needed]
+### Compute Infrastructure
+[More Information Needed]
+#### Hardware
+[More Information Needed]
+#### Software
+[More Information Needed]
+## Citation [optional]
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**BibTeX:**
+[More Information Needed]
+**APA:**
+[More Information Needed]
+## Glossary [optional]
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+[More Information Needed]
+## More Information [optional]
+[More Information Needed]
+## Model Card Authors [optional]
+[More Information Needed]
+## Model Card Contact
+[More Information Needed]
+### Framework versions
+- PEFT 0.19.1

checkpoint-1788/adapter_config.json ADDED Viewed

	@@ -0,0 +1,48 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "k_proj",
+    "up_proj",
+    "gate_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1788/adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:71ce8f09c288cc8c8b7ac6193695f7f42b8be0ba400b084b9163a8060c8df7bb
+size 80792880

checkpoint-1788/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,54 @@

+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-1788/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eb8661ae01cc3ee4014461498f74d402aa6d9cc3db91e59646e1f8d67b309f8b
+size 161810747

checkpoint-1788/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1dcad7f60928d68ad1f7bdc772f984932b0ff9aea4bcae90a976891265970458
+size 14645

checkpoint-1788/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:15bf83816ae439c3a890f09427b1f466dd2c99c9610a037a13b4a852802cadfe
+size 1465

checkpoint-1788/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

checkpoint-1788/tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,29 @@

+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-1788/trainer_state.json ADDED Viewed

	@@ -0,0 +1,744 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 1788,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6571732765436173,
+      "epoch": 0.013989437974329381,
+      "grad_norm": 0.0257568359375,
+      "learning_rate": 8.888888888888889e-05,
+      "loss": 1.9518862915039064,
+      "mean_token_accuracy": 0.5733682353049516,
+      "num_tokens": 294540.0,
+      "step": 25
+    },
+    {
+      "entropy": 1.7126372032612562,
+      "epoch": 0.027978875948658763,
+      "grad_norm": 0.01416015625,
+      "learning_rate": 0.0001814814814814815,
+      "loss": 1.6628182983398438,
+      "mean_token_accuracy": 0.6153687073290348,
+      "num_tokens": 593048.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.6262741280719637,
+      "epoch": 0.041968313922988144,
+      "grad_norm": 0.01806640625,
+      "learning_rate": 0.00019993435766282977,
+      "loss": 1.584449005126953,
+      "mean_token_accuracy": 0.6257701878249645,
+      "num_tokens": 891346.0,
+      "step": 75
+    },
+    {
+      "entropy": 1.58288999453187,
+      "epoch": 0.055957751897317526,
+      "grad_norm": 0.016357421875,
+      "learning_rate": 0.00019966783335941717,
+      "loss": 1.5507936096191406,
+      "mean_token_accuracy": 0.6367124810814857,
+      "num_tokens": 1184611.0,
+      "step": 100
+    },
+    {
+      "entropy": 1.5433334041386844,
+      "epoch": 0.0699471898716469,
+      "grad_norm": 0.0189208984375,
+      "learning_rate": 0.00019919687071661834,
+      "loss": 1.512821044921875,
+      "mean_token_accuracy": 0.6423260240256786,
+      "num_tokens": 1480125.0,
+      "step": 125
+    },
+    {
+      "entropy": 1.560064207613468,
+      "epoch": 0.08393662784597629,
+      "grad_norm": 0.017822265625,
+      "learning_rate": 0.00019852243577149215,
+      "loss": 1.520980224609375,
+      "mean_token_accuracy": 0.6398584572970867,
+      "num_tokens": 1770905.0,
+      "step": 150
+    },
+    {
+      "entropy": 1.4850163259357214,
+      "epoch": 0.09792606582030566,
+      "grad_norm": 0.0235595703125,
+      "learning_rate": 0.00019764591192282993,
+      "loss": 1.460587158203125,
+      "mean_token_accuracy": 0.6578983479738235,
+      "num_tokens": 2061126.0,
+      "step": 175
+    },
+    {
+      "entropy": 1.4848217015340923,
+      "epoch": 0.11191550379463505,
+      "grad_norm": 0.03662109375,
+      "learning_rate": 0.00019656909709353236,
+      "loss": 1.4420854187011718,
+      "mean_token_accuracy": 0.6592926564067603,
+      "num_tokens": 2352476.0,
+      "step": 200
+    },
+    {
+      "entropy": 1.4575298309139908,
+      "epoch": 0.12590494176896444,
+      "grad_norm": 0.0196533203125,
+      "learning_rate": 0.00019529420004271567,
+      "loss": 1.4135537719726563,
+      "mean_token_accuracy": 0.6653153513371944,
+      "num_tokens": 2647293.0,
+      "step": 225
+    },
+    {
+      "entropy": 1.440113589912653,
+      "epoch": 0.1398943797432938,
+      "grad_norm": 0.0238037109375,
+      "learning_rate": 0.00019382383583511206,
+      "loss": 1.416180877685547,
+      "mean_token_accuracy": 0.6659375876933337,
+      "num_tokens": 2941859.0,
+      "step": 250
+    },
+    {
+      "entropy": 1.4387904461845755,
+      "epoch": 0.1538838177176232,
+      "grad_norm": 0.0263671875,
+      "learning_rate": 0.00019216102047705746,
+      "loss": 1.38541015625,
+      "mean_token_accuracy": 0.6683934908360243,
+      "num_tokens": 3237763.0,
+      "step": 275
+    },
+    {
+      "entropy": 1.3664818671159447,
+      "epoch": 0.16787325569195258,
+      "grad_norm": 0.02001953125,
+      "learning_rate": 0.00019030916473006888,
+      "loss": 1.3284611511230469,
+      "mean_token_accuracy": 0.6843961907178163,
+      "num_tokens": 3541854.0,
+      "step": 300
+    },
+    {
+      "entropy": 1.3906436143629253,
+      "epoch": 0.18186269366628197,
+      "grad_norm": 0.0296630859375,
+      "learning_rate": 0.00018827206711470137,
+      "loss": 1.3411039733886718,
+      "mean_token_accuracy": 0.6773389867693186,
+      "num_tokens": 3832012.0,
+      "step": 325
+    },
+    {
+      "entropy": 1.3563544462341814,
+      "epoch": 0.19585213164061133,
+      "grad_norm": 0.0233154296875,
+      "learning_rate": 0.000186053906119035,
+      "loss": 1.3131784057617188,
+      "mean_token_accuracy": 0.686054406017065,
+      "num_tokens": 4129238.0,
+      "step": 350
+    },
+    {
+      "entropy": 1.3752087260596455,
+      "epoch": 0.20984156961494072,
+      "grad_norm": 0.0272216796875,
+      "learning_rate": 0.00018365923162777336,
+      "loss": 1.338323974609375,
+      "mean_token_accuracy": 0.6795472690463066,
+      "num_tokens": 4414939.0,
+      "step": 375
+    },
+    {
+      "entropy": 1.3106766785029322,
+      "epoch": 0.2238310075892701,
+      "grad_norm": 0.027099609375,
+      "learning_rate": 0.0001810929555895348,
+      "loss": 1.2676075744628905,
+      "mean_token_accuracy": 0.6990727451443672,
+      "num_tokens": 4711065.0,
+      "step": 400
+    },
+    {
+      "entropy": 1.2741564935538918,
+      "epoch": 0.2378204455635995,
+      "grad_norm": 0.0341796875,
+      "learning_rate": 0.00017836034194147922,
+      "loss": 1.231330337524414,
+      "mean_token_accuracy": 0.7052560625225306,
+      "num_tokens": 5006515.0,
+      "step": 425
+    },
+    {
+      "entropy": 1.255356255499646,
+      "epoch": 0.2518098835379289,
+      "grad_norm": 0.025390625,
+      "learning_rate": 0.000175466995811937,
+      "loss": 1.2256485748291015,
+      "mean_token_accuracy": 0.709921350106597,
+      "num_tokens": 5298743.0,
+      "step": 450
+    },
+    {
+      "entropy": 1.2404796612542122,
+      "epoch": 0.26579932151225827,
+      "grad_norm": 0.029296875,
+      "learning_rate": 0.00017241885202318787,
+      "loss": 1.199592056274414,
+      "mean_token_accuracy": 0.7124969442933797,
+      "num_tokens": 5587292.0,
+      "step": 475
+    },
+    {
+      "entropy": 1.2135492172930389,
+      "epoch": 0.2797887594865876,
+      "grad_norm": 0.0341796875,
+      "learning_rate": 0.00016922216291797248,
+      "loss": 1.1718869781494141,
+      "mean_token_accuracy": 0.7194690197706223,
+      "num_tokens": 5878179.0,
+      "step": 500
+    },
+    {
+      "entropy": 1.1519292463734745,
+      "epoch": 0.293778197460917,
+      "grad_norm": 0.044921875,
+      "learning_rate": 0.0001658834855347071,
+      "loss": 1.1038724517822265,
+      "mean_token_accuracy": 0.7360028678923846,
+      "num_tokens": 6174073.0,
+      "step": 525
+    },
+    {
+      "entropy": 1.2023496214020997,
+      "epoch": 0.3077676354352464,
+      "grad_norm": 0.035400390625,
+      "learning_rate": 0.00016240966815770754,
+      "loss": 1.1433139801025392,
+      "mean_token_accuracy": 0.7237949234992266,
+      "num_tokens": 6471377.0,
+      "step": 550
+    },
+    {
+      "entropy": 1.2287814150052145,
+      "epoch": 0.32175707340957577,
+      "grad_norm": 0.0262451171875,
+      "learning_rate": 0.00015880783627001026,
+      "loss": 1.196635513305664,
+      "mean_token_accuracy": 0.7167265053838492,
+      "num_tokens": 6762946.0,
+      "step": 575
+    },
+    {
+      "entropy": 1.1513058760762214,
+      "epoch": 0.33574651138390516,
+      "grad_norm": 0.03759765625,
+      "learning_rate": 0.0001550853779376045,
+      "loss": 1.104804229736328,
+      "mean_token_accuracy": 0.7350956991314888,
+      "num_tokens": 7058166.0,
+      "step": 600
+    },
+    {
+      "entropy": 1.1233506935788318,
+      "epoch": 0.34973594935823454,
+      "grad_norm": 0.038818359375,
+      "learning_rate": 0.00015124992865505523,
+      "loss": 1.0836825561523438,
+      "mean_token_accuracy": 0.7427010469138622,
+      "num_tokens": 7351236.0,
+      "step": 625
+    },
+    {
+      "entropy": 1.1373229866661132,
+      "epoch": 0.36372538733256393,
+      "grad_norm": 0.03466796875,
+      "learning_rate": 0.00014730935568360102,
+      "loss": 1.0919375610351563,
+      "mean_token_accuracy": 0.7399685283005237,
+      "num_tokens": 7642275.0,
+      "step": 650
+    },
+    {
+      "entropy": 1.1448556556599214,
+      "epoch": 0.3777148253068933,
+      "grad_norm": 0.033447265625,
+      "learning_rate": 0.0001432717419138532,
+      "loss": 1.0814331817626952,
+      "mean_token_accuracy": 0.7378807676583529,
+      "num_tokens": 7935222.0,
+      "step": 675
+    },
+    {
+      "entropy": 1.040786258596927,
+      "epoch": 0.39170426328122265,
+      "grad_norm": 0.0311279296875,
+      "learning_rate": 0.0001391453692861967,
+      "loss": 0.9921630096435546,
+      "mean_token_accuracy": 0.7638325883448124,
+      "num_tokens": 8234237.0,
+      "step": 700
+    },
+    {
+      "entropy": 1.1662991707585753,
+      "epoch": 0.40569370125555204,
+      "grad_norm": 0.0302734375,
+      "learning_rate": 0.0001349387018029003,
+      "loss": 1.118931884765625,
+      "mean_token_accuracy": 0.732994829416275,
+      "num_tokens": 8529663.0,
+      "step": 725
+    },
+    {
+      "entropy": 1.049061678093858,
+      "epoch": 0.41968313922988143,
+      "grad_norm": 0.041015625,
+      "learning_rate": 0.00013066036816678273,
+      "loss": 0.9903451538085938,
+      "mean_token_accuracy": 0.7613946237415076,
+      "num_tokens": 8817967.0,
+      "step": 750
+    },
+    {
+      "entropy": 1.1110075389547274,
+      "epoch": 0.4336725772042108,
+      "grad_norm": 0.0361328125,
+      "learning_rate": 0.0001263191440820448,
+      "loss": 1.0462547302246095,
+      "mean_token_accuracy": 0.7477244459837675,
+      "num_tokens": 9107363.0,
+      "step": 775
+    },
+    {
+      "entropy": 1.024681367701851,
+      "epoch": 0.4476620151785402,
+      "grad_norm": 0.039306640625,
+      "learning_rate": 0.00012192393425357353,
+      "loss": 0.9708639526367188,
+      "mean_token_accuracy": 0.7660154252499342,
+      "num_tokens": 9398385.0,
+      "step": 800
+    },
+    {
+      "entropy": 1.0914598937472328,
+      "epoch": 0.4616514531528696,
+      "grad_norm": 0.03662109375,
+      "learning_rate": 0.00011748375412163984,
+      "loss": 1.0285856628417969,
+      "mean_token_accuracy": 0.7533510192483663,
+      "num_tokens": 9692995.0,
+      "step": 825
+    },
+    {
+      "entropy": 1.070437773577869,
+      "epoch": 0.475640891127199,
+      "grad_norm": 0.0308837890625,
+      "learning_rate": 0.00011300771136945658,
+      "loss": 1.002013931274414,
+      "mean_token_accuracy": 0.7574464529007673,
+      "num_tokens": 9985540.0,
+      "step": 850
+    },
+    {
+      "entropy": 1.0500496013974772,
+      "epoch": 0.48963032910152837,
+      "grad_norm": 0.033935546875,
+      "learning_rate": 0.00010850498724152798,
+      "loss": 1.0061278533935547,
+      "mean_token_accuracy": 0.7601391483098269,
+      "num_tokens": 10280858.0,
+      "step": 875
+    },
+    {
+      "entropy": 0.9829169751703739,
+      "epoch": 0.5036197670758578,
+      "grad_norm": 0.035888671875,
+      "learning_rate": 0.00010398481771111037,
+      "loss": 0.9166652679443359,
+      "mean_token_accuracy": 0.7795024861395359,
+      "num_tokens": 10571194.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.9858249768940732,
+      "epoch": 0.5176092050501871,
+      "grad_norm": 0.038818359375,
+      "learning_rate": 9.94564745354137e-05,
+      "loss": 0.9250814819335937,
+      "mean_token_accuracy": 0.7783324559032917,
+      "num_tokens": 10861535.0,
+      "step": 925
+    },
+    {
+      "entropy": 0.9617049301974475,
+      "epoch": 0.5315986430245165,
+      "grad_norm": 0.0419921875,
+      "learning_rate": 9.492924623740271e-05,
+      "loss": 0.9093881988525391,
+      "mean_token_accuracy": 0.7818463468551635,
+      "num_tokens": 11157232.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.9786588902026415,
+      "epoch": 0.5455880809988459,
+      "grad_norm": 0.031494140625,
+      "learning_rate": 9.04124190532087e-05,
+      "loss": 0.9290876007080078,
+      "mean_token_accuracy": 0.7784507688879967,
+      "num_tokens": 11451817.0,
+      "step": 975
+    },
+    {
+      "entropy": 0.9941231621522456,
+      "epoch": 0.5595775189731752,
+      "grad_norm": 0.03662109375,
+      "learning_rate": 8.591525788423168e-05,
+      "loss": 0.9281739807128906,
+      "mean_token_accuracy": 0.7773911864310503,
+      "num_tokens": 11746337.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.9537221440812573,
+      "epoch": 0.5735669569475046,
+      "grad_norm": 0.0303955078125,
+      "learning_rate": 8.144698729300455e-05,
+      "loss": 0.8921097564697266,
+      "mean_token_accuracy": 0.7855567722022534,
+      "num_tokens": 12037412.0,
+      "step": 1025
+    },
+    {
+      "entropy": 0.9557448910363019,
+      "epoch": 0.587556394921834,
+      "grad_norm": 0.029052734375,
+      "learning_rate": 7.701677258179996e-05,
+      "loss": 0.9018702697753906,
+      "mean_token_accuracy": 0.78552112005651,
+      "num_tokens": 12330073.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.9831135825067758,
+      "epoch": 0.6015458328961634,
+      "grad_norm": 0.031005859375,
+      "learning_rate": 7.263370099279172e-05,
+      "loss": 0.9220442199707031,
+      "mean_token_accuracy": 0.7789029122143983,
+      "num_tokens": 12623027.0,
+      "step": 1075
+    },
+    {
+      "entropy": 0.8838035979354754,
+      "epoch": 0.6155352708704928,
+      "grad_norm": 0.040283203125,
+      "learning_rate": 6.83067630683331e-05,
+      "loss": 0.8227420043945313,
+      "mean_token_accuracy": 0.8038330339640379,
+      "num_tokens": 12916473.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.883046350381337,
+      "epoch": 0.6295247088448221,
+      "grad_norm": 0.037353515625,
+      "learning_rate": 6.404483420958494e-05,
+      "loss": 0.8318240356445312,
+      "mean_token_accuracy": 0.8026394218951464,
+      "num_tokens": 13210350.0,
+      "step": 1125
+    },
+    {
+      "entropy": 0.9277670627878979,
+      "epoch": 0.6435141468191515,
+      "grad_norm": 0.02783203125,
+      "learning_rate": 5.9856656471321636e-05,
+      "loss": 0.8780873870849609,
+      "mean_token_accuracy": 0.7957132039964199,
+      "num_tokens": 13501483.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.9232714768499136,
+      "epoch": 0.6575035847934809,
+      "grad_norm": 0.0255126953125,
+      "learning_rate": 5.575082063025619e-05,
+      "loss": 0.8550299835205079,
+      "mean_token_accuracy": 0.7936624947935342,
+      "num_tokens": 13797592.0,
+      "step": 1175
+    },
+    {
+      "entropy": 0.899801154313609,
+      "epoch": 0.6714930227678103,
+      "grad_norm": 0.03369140625,
+      "learning_rate": 5.173574856366683e-05,
+      "loss": 0.8404230499267578,
+      "mean_token_accuracy": 0.7989044986665249,
+      "num_tokens": 14085126.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.8731577204307541,
+      "epoch": 0.6854824607421397,
+      "grad_norm": 0.033203125,
+      "learning_rate": 4.7819675974469356e-05,
+      "loss": 0.8161344146728515,
+      "mean_token_accuracy": 0.8059349462389946,
+      "num_tokens": 14383100.0,
+      "step": 1225
+    },
+    {
+      "entropy": 0.9630948188621551,
+      "epoch": 0.6994718987164691,
+      "grad_norm": 0.033935546875,
+      "learning_rate": 4.401063549816984e-05,
+      "loss": 0.91357177734375,
+      "mean_token_accuracy": 0.7841827914863825,
+      "num_tokens": 14682819.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.9606740972073748,
+      "epoch": 0.7134613366907985,
+      "grad_norm": 0.02783203125,
+      "learning_rate": 4.031644022634831e-05,
+      "loss": 0.896214599609375,
+      "mean_token_accuracy": 0.7852722837030888,
+      "num_tokens": 14972956.0,
+      "step": 1275
+    },
+    {
+      "entropy": 0.9392211610358209,
+      "epoch": 0.7274507746651279,
+      "grad_norm": 0.033447265625,
+      "learning_rate": 3.674466768047078e-05,
+      "loss": 0.8826549530029297,
+      "mean_token_accuracy": 0.7912480696290731,
+      "num_tokens": 15271172.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.9369741760706529,
+      "epoch": 0.7414402126394573,
+      "grad_norm": 0.0228271484375,
+      "learning_rate": 3.330264426890114e-05,
+      "loss": 0.8716287994384766,
+      "mean_token_accuracy": 0.7906265539675951,
+      "num_tokens": 15567414.0,
+      "step": 1325
+    },
+    {
+      "entropy": 0.8475008888402954,
+      "epoch": 0.7554296506137866,
+      "grad_norm": 0.02734375,
+      "learning_rate": 2.9997430258996208e-05,
+      "loss": 0.7837786865234375,
+      "mean_token_accuracy": 0.8113828019797802,
+      "num_tokens": 15862493.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.8544112277310342,
+      "epoch": 0.7694190885881159,
+      "grad_norm": 0.0400390625,
+      "learning_rate": 2.6835805295107897e-05,
+      "loss": 0.7910999298095703,
+      "mean_token_accuracy": 0.8121292810887099,
+      "num_tokens": 16154298.0,
+      "step": 1375
+    },
+    {
+      "entropy": 0.8836554439552128,
+      "epoch": 0.7834085265624453,
+      "grad_norm": 0.03271484375,
+      "learning_rate": 2.382425449219854e-05,
+      "loss": 0.8322444152832031,
+      "mean_token_accuracy": 0.8058089179545641,
+      "num_tokens": 16450244.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.9750821837875993,
+      "epoch": 0.7973979645367747,
+      "grad_norm": 0.0279541015625,
+      "learning_rate": 2.0968955133593805e-05,
+      "loss": 0.9110565948486328,
+      "mean_token_accuracy": 0.7827954424917698,
+      "num_tokens": 16742734.0,
+      "step": 1425
+    },
+    {
+      "entropy": 0.8045650019776076,
+      "epoch": 0.8113874025111041,
+      "grad_norm": 0.037841796875,
+      "learning_rate": 1.8275764000159222e-05,
+      "loss": 0.7386186218261719,
+      "mean_token_accuracy": 0.8229204141348601,
+      "num_tokens": 17040960.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.9300361970160157,
+      "epoch": 0.8253768404854335,
+      "grad_norm": 0.0303955078125,
+      "learning_rate": 1.5750205356889937e-05,
+      "loss": 0.8793846130371094,
+      "mean_token_accuracy": 0.7934108532965183,
+      "num_tokens": 17340398.0,
+      "step": 1475
+    },
+    {
+      "entropy": 0.9426807796442881,
+      "epoch": 0.8393662784597629,
+      "grad_norm": 0.032958984375,
+      "learning_rate": 1.339745962155613e-05,
+      "loss": 0.8905538940429687,
+      "mean_token_accuracy": 0.7892987384647131,
+      "num_tokens": 17629025.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.8541876428946853,
+      "epoch": 0.8533557164340922,
+      "grad_norm": 0.02587890625,
+      "learning_rate": 1.1222352738646825e-05,
+      "loss": 0.7829645538330078,
+      "mean_token_accuracy": 0.8128130162507295,
+      "num_tokens": 17923764.0,
+      "step": 1525
+    },
+    {
+      "entropy": 0.9169456198904663,
+      "epoch": 0.8673451544084216,
+      "grad_norm": 0.0238037109375,
+      "learning_rate": 9.229346280407925e-06,
+      "loss": 0.8602067565917969,
+      "mean_token_accuracy": 0.7955809989571572,
+      "num_tokens": 18210238.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.8634558629477397,
+      "epoch": 0.881334592382751,
+      "grad_norm": 0.028564453125,
+      "learning_rate": 7.422528295279685e-06,
+      "loss": 0.8013089752197265,
+      "mean_token_accuracy": 0.8093983814865351,
+      "num_tokens": 18507153.0,
+      "step": 1575
+    },
+    {
+      "entropy": 0.9312617677496746,
+      "epoch": 0.8953240303570804,
+      "grad_norm": 0.037109375,
+      "learning_rate": 5.805604922504859e-06,
+      "loss": 0.8762725830078125,
+      "mean_token_accuracy": 0.791949690952897,
+      "num_tokens": 18804977.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.9295520845707506,
+      "epoch": 0.9093134683314098,
+      "grad_norm": 0.029541015625,
+      "learning_rate": 4.381892790107811e-06,
+      "loss": 0.8709014129638671,
+      "mean_token_accuracy": 0.7921189369261264,
+      "num_tokens": 19097011.0,
+      "step": 1625
+    },
+    {
+      "entropy": 0.9084588387049735,
+      "epoch": 0.9233029063057392,
+      "grad_norm": 0.036376953125,
+      "learning_rate": 3.154312211837673e-06,
+      "loss": 0.8461515045166016,
+      "mean_token_accuracy": 0.7987799559533596,
+      "num_tokens": 19394487.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.9174220200441777,
+      "epoch": 0.9372923442800686,
+      "grad_norm": 0.029541015625,
+      "learning_rate": 2.125381197030374e-06,
+      "loss": 0.8725605010986328,
+      "mean_token_accuracy": 0.7959611015766859,
+      "num_tokens": 19686426.0,
+      "step": 1675
+    },
+    {
+      "entropy": 0.9678884049877524,
+      "epoch": 0.951281782254398,
+      "grad_norm": 0.0264892578125,
+      "learning_rate": 1.297210285675754e-06,
+      "loss": 0.9031341552734375,
+      "mean_token_accuracy": 0.785126566067338,
+      "num_tokens": 19976869.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.9488437540875748,
+      "epoch": 0.9652712202287274,
+      "grad_norm": 0.03515625,
+      "learning_rate": 6.71498219284894e-07,
+      "loss": 0.8841629028320312,
+      "mean_token_accuracy": 0.7887707254290581,
+      "num_tokens": 20267550.0,
+      "step": 1725
+    },
+    {
+      "entropy": 0.9077194698620588,
+      "epoch": 0.9792606582030567,
+      "grad_norm": 0.037109375,
+      "learning_rate": 2.4952845643689827e-07,
+      "loss": 0.830882797241211,
+      "mean_token_accuracy": 0.7997597184032201,
+      "num_tokens": 20562898.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.8242502619419247,
+      "epoch": 0.993250096177386,
+      "grad_norm": 0.03076171875,
+      "learning_rate": 3.216654015283371e-08,
+      "loss": 0.7700749969482422,
+      "mean_token_accuracy": 0.8184793063253164,
+      "num_tokens": 20856654.0,
+      "step": 1775
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 1788,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.960165674320323e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1788/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b0ea67ee3810c6e10021bd35d99633fb742a8db898d131b220e180054e0dcd7
+size 5649

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,29 @@

+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b0ea67ee3810c6e10021bd35d99633fb742a8db898d131b220e180054e0dcd7
+size 5649