init

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5483b93231491dcdadb5963ea78a69366e0ab6aa03183648b5e5e587b20398e
+size 50792328

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/additional_config.json

@@ -0,0 +1 @@
+{"lora_dtype": null, "lorap_lr_ratio": null, "lorap_emb_lr": 1e-06}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/configuration.json

@@ -0,0 +1,11 @@
+{
+    "adapter_cfg": {
+        "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+        "model_revision": "main",
+        "sft_type": "lora",
+        "tuner_backend": "peft",
+        "template_type": "phi3-vl",
+        "dtype": "bf16",
+        "system": null
+    }
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 32000,
+  "max_new_tokens": 2048,
+  "pad_token_id": 32000,
+  "transformers_version": "4.46.3"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09ec37f8a196a72a97ca9967df7a30c712b3483c36593a578a3c2f921f48fb93
+size 100878458

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f0ebda40a3924e216542bcb94795169a0eced7e9d1672dd24ea3dc5bd826259
+size 14244

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1a5a353aa956358db56e4e8110f1cf929e76f1178ff0441174be62305b48b0e
+size 1064

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/sft_args.json

@@ -0,0 +1,246 @@
+{
+  "model_type": "phi3_5-vision-instruct",
+  "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "model_revision": "main",
+  "full_determinism": false,
+  "sft_type": "lora",
+  "freeze_parameters": [],
+  "freeze_vit": false,
+  "freeze_parameters_ratio": 0.0,
+  "additional_trainable_parameters": [],
+  "tuner_backend": "peft",
+  "template_type": "phi3-vl",
+  "output_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713",
+  "add_output_dir_suffix": true,
+  "ddp_backend": null,
+  "ddp_find_unused_parameters": null,
+  "ddp_broadcast_buffers": null,
+  "ddp_timeout": 1800,
+  "seed": 42,
+  "resume_from_checkpoint": null,
+  "resume_only_model": false,
+  "ignore_data_skip": false,
+  "dtype": "bf16",
+  "packing": false,
+  "train_backend": "transformers",
+  "tp": 1,
+  "pp": 1,
+  "min_lr": null,
+  "sequence_parallel": false,
+  "model_kwargs": {},
+  "loss_name": null,
+  "dataset": [
+    "llamafactory/OpenO1-SFT"
+  ],
+  "val_dataset": [],
+  "dataset_seed": 42,
+  "dataset_test_ratio": 0.01,
+  "use_loss_scale": false,
+  "loss_scale_config_path": "/home/pzcmoe/home/zhc/llama/swift/swift/llm/agent/default_loss_scale_config.json",
+  "system": null,
+  "tools_prompt": "react_en",
+  "max_length": 2048,
+  "truncation_strategy": "delete",
+  "check_dataset_strategy": "none",
+  "streaming": false,
+  "streaming_val_size": 0,
+  "streaming_buffer_size": 16384,
+  "model_name": [
+    null,
+    null
+  ],
+  "model_author": [
+    null,
+    null
+  ],
+  "quant_method": null,
+  "quantization_bit": 0,
+  "hqq_axis": 0,
+  "hqq_dynamic_config_path": null,
+  "bnb_4bit_comp_dtype": "bf16",
+  "bnb_4bit_quant_type": "nf4",
+  "bnb_4bit_use_double_quant": true,
+  "bnb_4bit_quant_storage": null,
+  "rescale_image": -1,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "target_regex": null,
+  "modules_to_save": [],
+  "lora_rank": 8,
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "lora_bias_trainable": "none",
+  "lora_dtype": null,
+  "lora_lr_ratio": null,
+  "use_rslora": false,
+  "use_dora": false,
+  "init_lora_weights": true,
+  "fourier_n_frequency": 2000,
+  "fourier_scaling": 300.0,
+  "rope_scaling": null,
+  "boft_block_size": 4,
+  "boft_block_num": 0,
+  "boft_n_butterfly_factor": 1,
+  "boft_dropout": 0.0,
+  "vera_rank": 256,
+  "vera_projection_prng_key": 0,
+  "vera_dropout": 0.0,
+  "vera_d_initial": 0.1,
+  "adapter_act": "gelu",
+  "adapter_length": 128,
+  "use_galore": false,
+  "galore_target_modules": null,
+  "galore_rank": 128,
+  "galore_update_proj_gap": 50,
+  "galore_scale": 1.0,
+  "galore_proj_type": "std",
+  "galore_optim_per_parameter": false,
+  "galore_with_embedding": false,
+  "galore_quantization": false,
+  "galore_proj_quant": false,
+  "galore_proj_bits": 4,
+  "galore_proj_group_size": 256,
+  "galore_cos_threshold": 0.4,
+  "galore_gamma_proj": 2,
+  "galore_queue_size": 5,
+  "adalora_target_r": 8,
+  "adalora_init_r": 12,
+  "adalora_tinit": 0,
+  "adalora_tfinal": 0,
+  "adalora_deltaT": 1,
+  "adalora_beta1": 0.85,
+  "adalora_beta2": 0.85,
+  "adalora_orth_reg_weight": 0.5,
+  "ia3_feedforward_modules": [],
+  "llamapro_num_new_blocks": 4,
+  "llamapro_num_groups": null,
+  "neftune_noise_alpha": null,
+  "neftune_backend": "transformers",
+  "lisa_activated_layers": 0,
+  "lisa_step_interval": 20,
+  "reft_layer_key": null,
+  "reft_layers": null,
+  "reft_rank": 4,
+  "reft_intervention_type": "LoreftIntervention",
+  "reft_args": null,
+  "use_liger": false,
+  "gradient_checkpointing": true,
+  "vit_use_gc": true,
+  "deepspeed": null,
+  "batch_size": 1,
+  "eval_batch_size": 1,
+  "auto_find_batch_size": false,
+  "num_train_epochs": 1,
+  "max_steps": 1000,
+  "optim": "adamw_torch",
+  "adam_beta1": 0.9,
+  "adam_beta2": 0.95,
+  "adam_epsilon": 1e-08,
+  "learning_rate": 0.0001,
+  "weight_decay": 0.1,
+  "gradient_accumulation_steps": 16,
+  "max_grad_norm": 1,
+  "predict_with_generate": false,
+  "lr_scheduler_type": "cosine",
+  "lr_scheduler_kwargs": {},
+  "warmup_ratio": 0.05,
+  "warmup_steps": 0,
+  "eval_steps": 50,
+  "save_steps": 50,
+  "save_only_model": false,
+  "save_total_limit": null,
+  "logging_steps": 5,
+  "acc_steps": 1,
+  "dataloader_num_workers": 1,
+  "dataloader_pin_memory": true,
+  "dataloader_drop_last": false,
+  "push_to_hub": false,
+  "hub_model_id": null,
+  "hub_token": null,
+  "hub_private_repo": false,
+  "hub_strategy": "every_save",
+  "test_oom_error": false,
+  "disable_tqdm": false,
+  "lazy_tokenize": true,
+  "preprocess_num_proc": 1,
+  "use_flash_attn": null,
+  "ignore_args_error": false,
+  "check_model_is_latest": true,
+  "logging_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs",
+  "report_to": [
+    "tensorboard"
+  ],
+  "acc_strategy": "token",
+  "save_on_each_node": false,
+  "evaluation_strategy": "steps",
+  "save_strategy": "steps",
+  "save_safetensors": true,
+  "gpu_memory_fraction": null,
+  "include_num_input_tokens_seen": false,
+  "local_repo_path": null,
+  "custom_register_path": null,
+  "custom_dataset_info": null,
+  "device_map_config": null,
+  "device_max_memory": [],
+  "max_new_tokens": 2048,
+  "do_sample": null,
+  "temperature": null,
+  "top_k": null,
+  "top_p": null,
+  "repetition_penalty": null,
+  "num_beams": 1,
+  "fsdp": "",
+  "fsdp_config": null,
+  "sequence_parallel_size": 1,
+  "model_layer_cls_name": null,
+  "metric_warmup_step": 0,
+  "fsdp_num": 1,
+  "per_device_train_batch_size": null,
+  "per_device_eval_batch_size": null,
+  "eval_strategy": null,
+  "self_cognition_sample": 0,
+  "train_dataset_mix_ratio": 0.0,
+  "train_dataset_mix_ds": [
+    "ms-bench"
+  ],
+  "train_dataset_sample": -1,
+  "val_dataset_sample": null,
+  "safe_serialization": null,
+  "only_save_model": null,
+  "neftune_alpha": null,
+  "deepspeed_config_path": null,
+  "model_cache_dir": null,
+  "lora_dropout_p": null,
+  "lora_target_modules": [],
+  "lora_target_regex": null,
+  "lora_modules_to_save": [],
+  "boft_target_modules": [],
+  "boft_modules_to_save": [],
+  "vera_target_modules": [],
+  "vera_modules_to_save": [],
+  "ia3_target_modules": [],
+  "ia3_modules_to_save": [],
+  "custom_train_dataset_path": [],
+  "custom_val_dataset_path": [],
+  "device_map_config_path": null,
+  "push_hub_strategy": null,
+  "use_self_cognition": false,
+  "is_multimodal": true,
+  "is_vision": true,
+  "lora_use_embedding": false,
+  "lora_use_all": false,
+  "lora_m2s_use_embedding": false,
+  "lora_m2s_use_ln": false,
+  "torch_dtype": "torch.bfloat16",
+  "fp16": false,
+  "bf16": true,
+  "rank": -1,
+  "local_rank": -1,
+  "world_size": 1,
+  "local_world_size": 1,
+  "bnb_4bit_compute_dtype": "torch.bfloat16",
+  "load_in_4bit": false,
+  "load_in_8bit": false,
+  "train_sampler_random": true,
+  "train_type": "sft",
+  "training_args": "Seq2SeqTrainingArguments(output_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, eval_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=1, per_device_eval_batch_size=1, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=16, eval_accumulation_steps=None, eval_delay=0, torch_empty_cache_steps=None, learning_rate=0.0001, weight_decay=0.1, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, max_grad_norm=1, num_train_epochs=1, max_steps=1000, lr_scheduler_type=<SchedulerType.COSINE: 'cosine'>, lr_scheduler_kwargs={}, warmup_ratio=0.05, warmup_steps=0, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=True, logging_steps=5, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=50, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=False, restore_callback_states_from_checkpoint=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=42, jit_mode_eval=False, use_ipex=False, bf16=True, fp16=False, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=50, dataloader_num_workers=1, dataloader_prefetch_factor=None, past_index=-1, run_name='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', disable_tqdm=False, remove_unused_columns=False, label_names=None, load_best_model_at_end=False, metric_for_best_model='loss', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, accelerator_config=AcceleratorConfig(split_batches=False, dispatch_batches=False, even_batches=True, use_seedable_sampler=True, non_blocking=False, gradient_accumulation_kwargs=None, use_configured_state=False), deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=False, resume_from_checkpoint=None, hub_model_id=None, hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, include_for_metrics=[], eval_do_concat_batches=True, fp16_backend='auto', evaluation_strategy=None, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=None, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=None, optim_target_modules=None, batch_eval_metrics=False, eval_on_start=False, use_liger_kernel=False, eval_use_gather_object=False, average_tokens_across_devices=False, sortish_sampler=False, predict_with_generate=False, generation_max_length=None, generation_num_beams=None, generation_config=GenerationConfig {\n  \"bos_token_id\": 1,\n  \"eos_token_id\": 32000,\n  \"max_new_tokens\": 2048,\n  \"pad_token_id\": 32000\n}\n, acc_strategy='token', loss_name=None, additional_saved_files=[], train_sampler_random=True, metric_warmup_step=0, train_dataset_sample=-1)"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/trainer_state.json

@@ -0,0 +1,261 @@
+{
+  "best_metric": 0.64535284,
+  "best_model_checkpoint": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100",
+  "epoch": 0.012837382457716871,
+  "eval_steps": 50,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "acc": 0.77952504,
+      "epoch": 0.0001283738245771687,
+      "grad_norm": 0.15511861443519592,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 0.82634771,
+      "memory(GiB)": 11.42,
+      "step": 1,
+      "train_speed(iter/s)": 0.092016
+    },
+    {
+      "acc": 0.77674872,
+      "epoch": 0.0006418691228858435,
+      "grad_norm": 0.14591480791568756,
+      "learning_rate": 1e-05,
+      "loss": 0.8159712,
+      "memory(GiB)": 12.15,
+      "step": 5,
+      "train_speed(iter/s)": 0.10552
+    },
+    {
+      "acc": 0.77379012,
+      "epoch": 0.001283738245771687,
+      "grad_norm": 0.151850163936615,
+      "learning_rate": 2e-05,
+      "loss": 0.8015564,
+      "memory(GiB)": 12.15,
+      "step": 10,
+      "train_speed(iter/s)": 0.107013
+    },
+    {
+      "acc": 0.7784133,
+      "epoch": 0.0019256073686575307,
+      "grad_norm": 0.15997682511806488,
+      "learning_rate": 3e-05,
+      "loss": 0.79284344,
+      "memory(GiB)": 12.15,
+      "step": 15,
+      "train_speed(iter/s)": 0.110995
+    },
+    {
+      "acc": 0.7815052,
+      "epoch": 0.002567476491543374,
+      "grad_norm": 0.14427673816680908,
+      "learning_rate": 4e-05,
+      "loss": 0.78122258,
+      "memory(GiB)": 12.9,
+      "step": 20,
+      "train_speed(iter/s)": 0.111462
+    },
+    {
+      "acc": 0.7915082,
+      "epoch": 0.0032093456144292177,
+      "grad_norm": 0.1590222716331482,
+      "learning_rate": 5e-05,
+      "loss": 0.7404335,
+      "memory(GiB)": 12.9,
+      "step": 25,
+      "train_speed(iter/s)": 0.112268
+    },
+    {
+      "acc": 0.80291214,
+      "epoch": 0.0038512147373150614,
+      "grad_norm": 0.13638992607593536,
+      "learning_rate": 6e-05,
+      "loss": 0.68063788,
+      "memory(GiB)": 12.9,
+      "step": 30,
+      "train_speed(iter/s)": 0.111517
+    },
+    {
+      "acc": 0.79923964,
+      "epoch": 0.0044930838602009054,
+      "grad_norm": 0.1271245777606964,
+      "learning_rate": 7e-05,
+      "loss": 0.68800621,
+      "memory(GiB)": 12.9,
+      "step": 35,
+      "train_speed(iter/s)": 0.110574
+    },
+    {
+      "acc": 0.80240812,
+      "epoch": 0.005134952983086748,
+      "grad_norm": 0.11937133222818375,
+      "learning_rate": 8e-05,
+      "loss": 0.69939818,
+      "memory(GiB)": 12.9,
+      "step": 40,
+      "train_speed(iter/s)": 0.109469
+    },
+    {
+      "acc": 0.78317132,
+      "epoch": 0.005776822105972592,
+      "grad_norm": 0.1665358543395996,
+      "learning_rate": 9e-05,
+      "loss": 0.75735736,
+      "memory(GiB)": 12.9,
+      "step": 45,
+      "train_speed(iter/s)": 0.108968
+    },
+    {
+      "acc": 0.80379133,
+      "epoch": 0.0064186912288584355,
+      "grad_norm": 0.14883440732955933,
+      "learning_rate": 0.0001,
+      "loss": 0.6824338,
+      "memory(GiB)": 12.9,
+      "step": 50,
+      "train_speed(iter/s)": 0.109163
+    },
+    {
+      "epoch": 0.0064186912288584355,
+      "eval_acc": 0.8081346555994846,
+      "eval_loss": 0.6729312539100647,
+      "eval_runtime": 195.4613,
+      "eval_samples_per_second": 6.436,
+      "eval_steps_per_second": 6.436,
+      "step": 50
+    },
+    {
+      "acc": 0.7984889,
+      "epoch": 0.007060560351744279,
+      "grad_norm": 0.16687534749507904,
+      "learning_rate": 9.999316524962345e-05,
+      "loss": 0.7141089,
+      "memory(GiB)": 12.9,
+      "step": 55,
+      "train_speed(iter/s)": 0.078646
+    },
+    {
+      "acc": 0.79216175,
+      "epoch": 0.007702429474630123,
+      "grad_norm": 0.17546051740646362,
+      "learning_rate": 9.997266286704631e-05,
+      "loss": 0.72419305,
+      "memory(GiB)": 12.9,
+      "step": 60,
+      "train_speed(iter/s)": 0.080466
+    },
+    {
+      "acc": 0.80358973,
+      "epoch": 0.008344298597515967,
+      "grad_norm": 0.16079305112361908,
+      "learning_rate": 9.993849845741524e-05,
+      "loss": 0.67717872,
+      "memory(GiB)": 12.9,
+      "step": 65,
+      "train_speed(iter/s)": 0.082252
+    },
+    {
+      "acc": 0.80615339,
+      "epoch": 0.008986167720401811,
+      "grad_norm": 0.1396234929561615,
+      "learning_rate": 9.989068136093873e-05,
+      "loss": 0.70638905,
+      "memory(GiB)": 12.9,
+      "step": 70,
+      "train_speed(iter/s)": 0.084
+    },
+    {
+      "acc": 0.79676814,
+      "epoch": 0.009628036843287653,
+      "grad_norm": 0.1340462565422058,
+      "learning_rate": 9.98292246503335e-05,
+      "loss": 0.71420946,
+      "memory(GiB)": 12.9,
+      "step": 75,
+      "train_speed(iter/s)": 0.08542
+    },
+    {
+      "acc": 0.79325166,
+      "epoch": 0.010269905966173496,
+      "grad_norm": 0.13259001076221466,
+      "learning_rate": 9.975414512725057e-05,
+      "loss": 0.71777782,
+      "memory(GiB)": 12.9,
+      "step": 80,
+      "train_speed(iter/s)": 0.086655
+    },
+    {
+      "acc": 0.81705523,
+      "epoch": 0.01091177508905934,
+      "grad_norm": 0.15833652019500732,
+      "learning_rate": 9.966546331768191e-05,
+      "loss": 0.62526927,
+      "memory(GiB)": 12.9,
+      "step": 85,
+      "train_speed(iter/s)": 0.088014
+    },
+    {
+      "acc": 0.80851898,
+      "epoch": 0.011553644211945184,
+      "grad_norm": 0.1456330418586731,
+      "learning_rate": 9.956320346634876e-05,
+      "loss": 0.68010745,
+      "memory(GiB)": 12.9,
+      "step": 90,
+      "train_speed(iter/s)": 0.088941
+    },
+    {
+      "acc": 0.81903477,
+      "epoch": 0.012195513334831027,
+      "grad_norm": 0.14285945892333984,
+      "learning_rate": 9.944739353007344e-05,
+      "loss": 0.6090106,
+      "memory(GiB)": 12.9,
+      "step": 95,
+      "train_speed(iter/s)": 0.089902
+    },
+    {
+      "acc": 0.82483721,
+      "epoch": 0.012837382457716871,
+      "grad_norm": 0.1478744000196457,
+      "learning_rate": 9.931806517013612e-05,
+      "loss": 0.59807158,
+      "memory(GiB)": 12.9,
+      "step": 100,
+      "train_speed(iter/s)": 0.090663
+    },
+    {
+      "epoch": 0.012837382457716871,
+      "eval_acc": 0.813768508772143,
+      "eval_loss": 0.645352840423584,
+      "eval_runtime": 195.755,
+      "eval_samples_per_second": 6.426,
+      "eval_steps_per_second": 6.426,
+      "step": 100
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.887966208382157e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5ca841f7bd4a855ee575421abc61a664c4a5bac12d5a654d3adfe5e4f1cc45d
+size 7480

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02b047ba6ad4ac3fc6b0980080d4d3c0cf69641417e5b7ac54d06a2e96ba9eea
+size 50792328

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/additional_config.json

@@ -0,0 +1 @@
+{"lora_dtype": null, "lorap_lr_ratio": null, "lorap_emb_lr": 1e-06}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/configuration.json

@@ -0,0 +1,11 @@
+{
+    "adapter_cfg": {
+        "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+        "model_revision": "main",
+        "sft_type": "lora",
+        "tuner_backend": "peft",
+        "template_type": "phi3-vl",
+        "dtype": "bf16",
+        "system": null
+    }
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 32000,
+  "max_new_tokens": 2048,
+  "pad_token_id": 32000,
+  "transformers_version": "4.46.3"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d13564f1b33e65744b4ae00df396b38868578d8aedd49db3aac4d78b0697067
+size 100878458

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b78ad66f92e3ca930b0c6b16fe26286bf9120471a4b51538be0d69091971a43
+size 14244

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4077036d99500a708f700f75da24d51b5300e184ad35fda49dc5a4df5596cca2
+size 1064

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/sft_args.json

@@ -0,0 +1,246 @@
+{
+  "model_type": "phi3_5-vision-instruct",
+  "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "model_revision": "main",
+  "full_determinism": false,
+  "sft_type": "lora",
+  "freeze_parameters": [],
+  "freeze_vit": false,
+  "freeze_parameters_ratio": 0.0,
+  "additional_trainable_parameters": [],
+  "tuner_backend": "peft",
+  "template_type": "phi3-vl",
+  "output_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713",
+  "add_output_dir_suffix": true,
+  "ddp_backend": null,
+  "ddp_find_unused_parameters": null,
+  "ddp_broadcast_buffers": null,
+  "ddp_timeout": 1800,
+  "seed": 42,
+  "resume_from_checkpoint": null,
+  "resume_only_model": false,
+  "ignore_data_skip": false,
+  "dtype": "bf16",
+  "packing": false,
+  "train_backend": "transformers",
+  "tp": 1,
+  "pp": 1,
+  "min_lr": null,
+  "sequence_parallel": false,
+  "model_kwargs": {},
+  "loss_name": null,
+  "dataset": [
+    "llamafactory/OpenO1-SFT"
+  ],
+  "val_dataset": [],
+  "dataset_seed": 42,
+  "dataset_test_ratio": 0.01,
+  "use_loss_scale": false,
+  "loss_scale_config_path": "/home/pzcmoe/home/zhc/llama/swift/swift/llm/agent/default_loss_scale_config.json",
+  "system": null,
+  "tools_prompt": "react_en",
+  "max_length": 2048,
+  "truncation_strategy": "delete",
+  "check_dataset_strategy": "none",
+  "streaming": false,
+  "streaming_val_size": 0,
+  "streaming_buffer_size": 16384,
+  "model_name": [
+    null,
+    null
+  ],
+  "model_author": [
+    null,
+    null
+  ],
+  "quant_method": null,
+  "quantization_bit": 0,
+  "hqq_axis": 0,
+  "hqq_dynamic_config_path": null,
+  "bnb_4bit_comp_dtype": "bf16",
+  "bnb_4bit_quant_type": "nf4",
+  "bnb_4bit_use_double_quant": true,
+  "bnb_4bit_quant_storage": null,
+  "rescale_image": -1,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "target_regex": null,
+  "modules_to_save": [],
+  "lora_rank": 8,
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "lora_bias_trainable": "none",
+  "lora_dtype": null,
+  "lora_lr_ratio": null,
+  "use_rslora": false,
+  "use_dora": false,
+  "init_lora_weights": true,
+  "fourier_n_frequency": 2000,
+  "fourier_scaling": 300.0,
+  "rope_scaling": null,
+  "boft_block_size": 4,
+  "boft_block_num": 0,
+  "boft_n_butterfly_factor": 1,
+  "boft_dropout": 0.0,
+  "vera_rank": 256,
+  "vera_projection_prng_key": 0,
+  "vera_dropout": 0.0,
+  "vera_d_initial": 0.1,
+  "adapter_act": "gelu",
+  "adapter_length": 128,
+  "use_galore": false,
+  "galore_target_modules": null,
+  "galore_rank": 128,
+  "galore_update_proj_gap": 50,
+  "galore_scale": 1.0,
+  "galore_proj_type": "std",
+  "galore_optim_per_parameter": false,
+  "galore_with_embedding": false,
+  "galore_quantization": false,
+  "galore_proj_quant": false,
+  "galore_proj_bits": 4,
+  "galore_proj_group_size": 256,
+  "galore_cos_threshold": 0.4,
+  "galore_gamma_proj": 2,
+  "galore_queue_size": 5,
+  "adalora_target_r": 8,
+  "adalora_init_r": 12,
+  "adalora_tinit": 0,
+  "adalora_tfinal": 0,
+  "adalora_deltaT": 1,
+  "adalora_beta1": 0.85,
+  "adalora_beta2": 0.85,
+  "adalora_orth_reg_weight": 0.5,
+  "ia3_feedforward_modules": [],
+  "llamapro_num_new_blocks": 4,
+  "llamapro_num_groups": null,
+  "neftune_noise_alpha": null,
+  "neftune_backend": "transformers",
+  "lisa_activated_layers": 0,
+  "lisa_step_interval": 20,
+  "reft_layer_key": null,
+  "reft_layers": null,
+  "reft_rank": 4,
+  "reft_intervention_type": "LoreftIntervention",
+  "reft_args": null,
+  "use_liger": false,
+  "gradient_checkpointing": true,
+  "vit_use_gc": true,
+  "deepspeed": null,
+  "batch_size": 1,
+  "eval_batch_size": 1,
+  "auto_find_batch_size": false,
+  "num_train_epochs": 1,
+  "max_steps": 1000,
+  "optim": "adamw_torch",
+  "adam_beta1": 0.9,
+  "adam_beta2": 0.95,
+  "adam_epsilon": 1e-08,
+  "learning_rate": 0.0001,
+  "weight_decay": 0.1,
+  "gradient_accumulation_steps": 16,
+  "max_grad_norm": 1,
+  "predict_with_generate": false,
+  "lr_scheduler_type": "cosine",
+  "lr_scheduler_kwargs": {},
+  "warmup_ratio": 0.05,
+  "warmup_steps": 0,
+  "eval_steps": 50,
+  "save_steps": 50,
+  "save_only_model": false,
+  "save_total_limit": null,
+  "logging_steps": 5,
+  "acc_steps": 1,
+  "dataloader_num_workers": 1,
+  "dataloader_pin_memory": true,
+  "dataloader_drop_last": false,
+  "push_to_hub": false,
+  "hub_model_id": null,
+  "hub_token": null,
+  "hub_private_repo": false,
+  "hub_strategy": "every_save",
+  "test_oom_error": false,
+  "disable_tqdm": false,
+  "lazy_tokenize": true,
+  "preprocess_num_proc": 1,
+  "use_flash_attn": null,
+  "ignore_args_error": false,
+  "check_model_is_latest": true,
+  "logging_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs",
+  "report_to": [
+    "tensorboard"
+  ],
+  "acc_strategy": "token",
+  "save_on_each_node": false,
+  "evaluation_strategy": "steps",
+  "save_strategy": "steps",
+  "save_safetensors": true,
+  "gpu_memory_fraction": null,
+  "include_num_input_tokens_seen": false,
+  "local_repo_path": null,
+  "custom_register_path": null,
+  "custom_dataset_info": null,
+  "device_map_config": null,
+  "device_max_memory": [],
+  "max_new_tokens": 2048,
+  "do_sample": null,
+  "temperature": null,
+  "top_k": null,
+  "top_p": null,
+  "repetition_penalty": null,
+  "num_beams": 1,
+  "fsdp": "",
+  "fsdp_config": null,
+  "sequence_parallel_size": 1,
+  "model_layer_cls_name": null,
+  "metric_warmup_step": 0,
+  "fsdp_num": 1,
+  "per_device_train_batch_size": null,
+  "per_device_eval_batch_size": null,
+  "eval_strategy": null,
+  "self_cognition_sample": 0,
+  "train_dataset_mix_ratio": 0.0,
+  "train_dataset_mix_ds": [
+    "ms-bench"
+  ],
+  "train_dataset_sample": -1,
+  "val_dataset_sample": null,
+  "safe_serialization": null,
+  "only_save_model": null,
+  "neftune_alpha": null,
+  "deepspeed_config_path": null,
+  "model_cache_dir": null,
+  "lora_dropout_p": null,
+  "lora_target_modules": [],
+  "lora_target_regex": null,
+  "lora_modules_to_save": [],
+  "boft_target_modules": [],
+  "boft_modules_to_save": [],
+  "vera_target_modules": [],
+  "vera_modules_to_save": [],
+  "ia3_target_modules": [],
+  "ia3_modules_to_save": [],
+  "custom_train_dataset_path": [],
+  "custom_val_dataset_path": [],
+  "device_map_config_path": null,
+  "push_hub_strategy": null,
+  "use_self_cognition": false,
+  "is_multimodal": true,
+  "is_vision": true,
+  "lora_use_embedding": false,
+  "lora_use_all": false,
+  "lora_m2s_use_embedding": false,
+  "lora_m2s_use_ln": false,
+  "torch_dtype": "torch.bfloat16",
+  "fp16": false,
+  "bf16": true,
+  "rank": -1,
+  "local_rank": -1,
+  "world_size": 1,
+  "local_world_size": 1,
+  "bnb_4bit_compute_dtype": "torch.bfloat16",
+  "load_in_4bit": false,
+  "load_in_8bit": false,
+  "train_sampler_random": true,
+  "train_type": "sft",
+  "training_args": "Seq2SeqTrainingArguments(output_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, eval_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=1, per_device_eval_batch_size=1, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=16, eval_accumulation_steps=None, eval_delay=0, torch_empty_cache_steps=None, learning_rate=0.0001, weight_decay=0.1, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, max_grad_norm=1, num_train_epochs=1, max_steps=1000, lr_scheduler_type=<SchedulerType.COSINE: 'cosine'>, lr_scheduler_kwargs={}, warmup_ratio=0.05, warmup_steps=0, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=True, logging_steps=5, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=50, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=False, restore_callback_states_from_checkpoint=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=42, jit_mode_eval=False, use_ipex=False, bf16=True, fp16=False, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=50, dataloader_num_workers=1, dataloader_prefetch_factor=None, past_index=-1, run_name='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', disable_tqdm=False, remove_unused_columns=False, label_names=None, load_best_model_at_end=False, metric_for_best_model='loss', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, accelerator_config=AcceleratorConfig(split_batches=False, dispatch_batches=False, even_batches=True, use_seedable_sampler=True, non_blocking=False, gradient_accumulation_kwargs=None, use_configured_state=False), deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=False, resume_from_checkpoint=None, hub_model_id=None, hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, include_for_metrics=[], eval_do_concat_batches=True, fp16_backend='auto', evaluation_strategy=None, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=None, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=None, optim_target_modules=None, batch_eval_metrics=False, eval_on_start=False, use_liger_kernel=False, eval_use_gather_object=False, average_tokens_across_devices=False, sortish_sampler=False, predict_with_generate=False, generation_max_length=None, generation_num_beams=None, generation_config=GenerationConfig {\n  \"bos_token_id\": 1,\n  \"eos_token_id\": 32000,\n  \"max_new_tokens\": 2048,\n  \"pad_token_id\": 32000\n}\n, acc_strategy='token', loss_name=None, additional_saved_files=[], train_sampler_random=True, metric_warmup_step=0, train_dataset_sample=-1)"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/trainer_state.json

@@ -0,0 +1,2223 @@
+{
+  "best_metric": 0.60646284,
+  "best_model_checkpoint": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/checkpoint-850",
+  "epoch": 0.12837382457716873,
+  "eval_steps": 50,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "acc": 0.77952504,
+      "epoch": 0.0001283738245771687,
+      "grad_norm": 0.15511861443519592,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 0.82634771,
+      "memory(GiB)": 11.42,
+      "step": 1,
+      "train_speed(iter/s)": 0.092016
+    },
+    {
+      "acc": 0.77674872,
+      "epoch": 0.0006418691228858435,
+      "grad_norm": 0.14591480791568756,
+      "learning_rate": 1e-05,
+      "loss": 0.8159712,
+      "memory(GiB)": 12.15,
+      "step": 5,
+      "train_speed(iter/s)": 0.10552
+    },
+    {
+      "acc": 0.77379012,
+      "epoch": 0.001283738245771687,
+      "grad_norm": 0.151850163936615,
+      "learning_rate": 2e-05,
+      "loss": 0.8015564,
+      "memory(GiB)": 12.15,
+      "step": 10,
+      "train_speed(iter/s)": 0.107013
+    },
+    {
+      "acc": 0.7784133,
+      "epoch": 0.0019256073686575307,
+      "grad_norm": 0.15997682511806488,
+      "learning_rate": 3e-05,
+      "loss": 0.79284344,
+      "memory(GiB)": 12.15,
+      "step": 15,
+      "train_speed(iter/s)": 0.110995
+    },
+    {
+      "acc": 0.7815052,
+      "epoch": 0.002567476491543374,
+      "grad_norm": 0.14427673816680908,
+      "learning_rate": 4e-05,
+      "loss": 0.78122258,
+      "memory(GiB)": 12.9,
+      "step": 20,
+      "train_speed(iter/s)": 0.111462
+    },
+    {
+      "acc": 0.7915082,
+      "epoch": 0.0032093456144292177,
+      "grad_norm": 0.1590222716331482,
+      "learning_rate": 5e-05,
+      "loss": 0.7404335,
+      "memory(GiB)": 12.9,
+      "step": 25,
+      "train_speed(iter/s)": 0.112268
+    },
+    {
+      "acc": 0.80291214,
+      "epoch": 0.0038512147373150614,
+      "grad_norm": 0.13638992607593536,
+      "learning_rate": 6e-05,
+      "loss": 0.68063788,
+      "memory(GiB)": 12.9,
+      "step": 30,
+      "train_speed(iter/s)": 0.111517
+    },
+    {
+      "acc": 0.79923964,
+      "epoch": 0.0044930838602009054,
+      "grad_norm": 0.1271245777606964,
+      "learning_rate": 7e-05,
+      "loss": 0.68800621,
+      "memory(GiB)": 12.9,
+      "step": 35,
+      "train_speed(iter/s)": 0.110574
+    },
+    {
+      "acc": 0.80240812,
+      "epoch": 0.005134952983086748,
+      "grad_norm": 0.11937133222818375,
+      "learning_rate": 8e-05,
+      "loss": 0.69939818,
+      "memory(GiB)": 12.9,
+      "step": 40,
+      "train_speed(iter/s)": 0.109469
+    },
+    {
+      "acc": 0.78317132,
+      "epoch": 0.005776822105972592,
+      "grad_norm": 0.1665358543395996,
+      "learning_rate": 9e-05,
+      "loss": 0.75735736,
+      "memory(GiB)": 12.9,
+      "step": 45,
+      "train_speed(iter/s)": 0.108968
+    },
+    {
+      "acc": 0.80379133,
+      "epoch": 0.0064186912288584355,
+      "grad_norm": 0.14883440732955933,
+      "learning_rate": 0.0001,
+      "loss": 0.6824338,
+      "memory(GiB)": 12.9,
+      "step": 50,
+      "train_speed(iter/s)": 0.109163
+    },
+    {
+      "epoch": 0.0064186912288584355,
+      "eval_acc": 0.8081346555994846,
+      "eval_loss": 0.6729312539100647,
+      "eval_runtime": 195.4613,
+      "eval_samples_per_second": 6.436,
+      "eval_steps_per_second": 6.436,
+      "step": 50
+    },
+    {
+      "acc": 0.7984889,
+      "epoch": 0.007060560351744279,
+      "grad_norm": 0.16687534749507904,
+      "learning_rate": 9.999316524962345e-05,
+      "loss": 0.7141089,
+      "memory(GiB)": 12.9,
+      "step": 55,
+      "train_speed(iter/s)": 0.078646
+    },
+    {
+      "acc": 0.79216175,
+      "epoch": 0.007702429474630123,
+      "grad_norm": 0.17546051740646362,
+      "learning_rate": 9.997266286704631e-05,
+      "loss": 0.72419305,
+      "memory(GiB)": 12.9,
+      "step": 60,
+      "train_speed(iter/s)": 0.080466
+    },
+    {
+      "acc": 0.80358973,
+      "epoch": 0.008344298597515967,
+      "grad_norm": 0.16079305112361908,
+      "learning_rate": 9.993849845741524e-05,
+      "loss": 0.67717872,
+      "memory(GiB)": 12.9,
+      "step": 65,
+      "train_speed(iter/s)": 0.082252
+    },
+    {
+      "acc": 0.80615339,
+      "epoch": 0.008986167720401811,
+      "grad_norm": 0.1396234929561615,
+      "learning_rate": 9.989068136093873e-05,
+      "loss": 0.70638905,
+      "memory(GiB)": 12.9,
+      "step": 70,
+      "train_speed(iter/s)": 0.084
+    },
+    {
+      "acc": 0.79676814,
+      "epoch": 0.009628036843287653,
+      "grad_norm": 0.1340462565422058,
+      "learning_rate": 9.98292246503335e-05,
+      "loss": 0.71420946,
+      "memory(GiB)": 12.9,
+      "step": 75,
+      "train_speed(iter/s)": 0.08542
+    },
+    {
+      "acc": 0.79325166,
+      "epoch": 0.010269905966173496,
+      "grad_norm": 0.13259001076221466,
+      "learning_rate": 9.975414512725057e-05,
+      "loss": 0.71777782,
+      "memory(GiB)": 12.9,
+      "step": 80,
+      "train_speed(iter/s)": 0.086655
+    },
+    {
+      "acc": 0.81705523,
+      "epoch": 0.01091177508905934,
+      "grad_norm": 0.15833652019500732,
+      "learning_rate": 9.966546331768191e-05,
+      "loss": 0.62526927,
+      "memory(GiB)": 12.9,
+      "step": 85,
+      "train_speed(iter/s)": 0.088014
+    },
+    {
+      "acc": 0.80851898,
+      "epoch": 0.011553644211945184,
+      "grad_norm": 0.1456330418586731,
+      "learning_rate": 9.956320346634876e-05,
+      "loss": 0.68010745,
+      "memory(GiB)": 12.9,
+      "step": 90,
+      "train_speed(iter/s)": 0.088941
+    },
+    {
+      "acc": 0.81903477,
+      "epoch": 0.012195513334831027,
+      "grad_norm": 0.14285945892333984,
+      "learning_rate": 9.944739353007344e-05,
+      "loss": 0.6090106,
+      "memory(GiB)": 12.9,
+      "step": 95,
+      "train_speed(iter/s)": 0.089902
+    },
+    {
+      "acc": 0.82483721,
+      "epoch": 0.012837382457716871,
+      "grad_norm": 0.1478744000196457,
+      "learning_rate": 9.931806517013612e-05,
+      "loss": 0.59807158,
+      "memory(GiB)": 12.9,
+      "step": 100,
+      "train_speed(iter/s)": 0.090663
+    },
+    {
+      "epoch": 0.012837382457716871,
+      "eval_acc": 0.813768508772143,
+      "eval_loss": 0.645352840423584,
+      "eval_runtime": 195.755,
+      "eval_samples_per_second": 6.426,
+      "eval_steps_per_second": 6.426,
+      "step": 100
+    },
+    {
+      "acc": 0.80671978,
+      "epoch": 0.013479251580602715,
+      "grad_norm": 0.15444877743721008,
+      "learning_rate": 9.917525374361912e-05,
+      "loss": 0.66537051,
+      "memory(GiB)": 12.9,
+      "step": 105,
+      "train_speed(iter/s)": 0.078062
+    },
+    {
+      "acc": 0.81962786,
+      "epoch": 0.014121120703488558,
+      "grad_norm": 0.16826578974723816,
+      "learning_rate": 9.901899829374047e-05,
+      "loss": 0.60835891,
+      "memory(GiB)": 12.9,
+      "step": 110,
+      "train_speed(iter/s)": 0.079089
+    },
+    {
+      "acc": 0.80644588,
+      "epoch": 0.014762989826374402,
+      "grad_norm": 0.1519591063261032,
+      "learning_rate": 9.884934153917997e-05,
+      "loss": 0.66763787,
+      "memory(GiB)": 12.9,
+      "step": 115,
+      "train_speed(iter/s)": 0.08012
+    },
+    {
+      "acc": 0.8235981,
+      "epoch": 0.015404858949260246,
+      "grad_norm": 0.20040030777454376,
+      "learning_rate": 9.86663298624003e-05,
+      "loss": 0.5919826,
+      "memory(GiB)": 12.9,
+      "step": 120,
+      "train_speed(iter/s)": 0.081125
+    },
+    {
+      "acc": 0.81199884,
+      "epoch": 0.01604672807214609,
+      "grad_norm": 0.1842738389968872,
+      "learning_rate": 9.847001329696653e-05,
+      "loss": 0.63266978,
+      "memory(GiB)": 12.9,
+      "step": 125,
+      "train_speed(iter/s)": 0.082093
+    },
+    {
+      "acc": 0.81563005,
+      "epoch": 0.016688597195031935,
+      "grad_norm": 0.15375380218029022,
+      "learning_rate": 9.826044551386744e-05,
+      "loss": 0.61709371,
+      "memory(GiB)": 12.9,
+      "step": 130,
+      "train_speed(iter/s)": 0.082937
+    },
+    {
+      "acc": 0.80918331,
+      "epoch": 0.017330466317917778,
+      "grad_norm": 0.14841115474700928,
+      "learning_rate": 9.803768380684242e-05,
+      "loss": 0.63595695,
+      "memory(GiB)": 12.9,
+      "step": 135,
+      "train_speed(iter/s)": 0.083738
+    },
+    {
+      "acc": 0.81229877,
+      "epoch": 0.017972335440803622,
+      "grad_norm": 0.16627077758312225,
+      "learning_rate": 9.780178907671789e-05,
+      "loss": 0.64757662,
+      "memory(GiB)": 12.9,
+      "step": 140,
+      "train_speed(iter/s)": 0.084642
+    },
+    {
+      "acc": 0.804142,
+      "epoch": 0.018614204563689465,
+      "grad_norm": 0.1714244931936264,
+      "learning_rate": 9.755282581475769e-05,
+      "loss": 0.6832706,
+      "memory(GiB)": 12.9,
+      "step": 145,
+      "train_speed(iter/s)": 0.085406
+    },
+    {
+      "acc": 0.79688311,
+      "epoch": 0.019256073686575306,
+      "grad_norm": 0.18916593492031097,
+      "learning_rate": 9.729086208503174e-05,
+      "loss": 0.70668921,
+      "memory(GiB)": 12.9,
+      "step": 150,
+      "train_speed(iter/s)": 0.086183
+    },
+    {
+      "epoch": 0.019256073686575306,
+      "eval_acc": 0.8153510047395746,
+      "eval_loss": 0.6350318193435669,
+      "eval_runtime": 196.0265,
+      "eval_samples_per_second": 6.417,
+      "eval_steps_per_second": 6.417,
+      "step": 150
+    },
+    {
+      "acc": 0.80884809,
+      "epoch": 0.01989794280946115,
+      "grad_norm": 0.17633555829524994,
+      "learning_rate": 9.701596950580806e-05,
+      "loss": 0.63387117,
+      "memory(GiB)": 12.9,
+      "step": 155,
+      "train_speed(iter/s)": 0.078254
+    },
+    {
+      "acc": 0.81135025,
+      "epoch": 0.020539811932346993,
+      "grad_norm": 0.18200528621673584,
+      "learning_rate": 9.672822322997305e-05,
+      "loss": 0.65075536,
+      "memory(GiB)": 12.9,
+      "step": 160,
+      "train_speed(iter/s)": 0.078985
+    },
+    {
+      "acc": 0.82410259,
+      "epoch": 0.021181681055232836,
+      "grad_norm": 0.14853200316429138,
+      "learning_rate": 9.642770192448536e-05,
+      "loss": 0.60177903,
+      "memory(GiB)": 12.9,
+      "step": 165,
+      "train_speed(iter/s)": 0.079746
+    },
+    {
+      "acc": 0.80189457,
+      "epoch": 0.02182355017811868,
+      "grad_norm": 0.17732569575309753,
+      "learning_rate": 9.611448774886924e-05,
+      "loss": 0.68190188,
+      "memory(GiB)": 12.9,
+      "step": 170,
+      "train_speed(iter/s)": 0.080445
+    },
+    {
+      "acc": 0.81763601,
+      "epoch": 0.022465419301004524,
+      "grad_norm": 0.3583473563194275,
+      "learning_rate": 9.578866633275288e-05,
+      "loss": 0.62211475,
+      "memory(GiB)": 12.9,
+      "step": 175,
+      "train_speed(iter/s)": 0.081173
+    },
+    {
+      "acc": 0.81451321,
+      "epoch": 0.023107288423890367,
+      "grad_norm": 0.19369152188301086,
+      "learning_rate": 9.545032675245813e-05,
+      "loss": 0.63143706,
+      "memory(GiB)": 12.9,
+      "step": 180,
+      "train_speed(iter/s)": 0.081877
+    },
+    {
+      "acc": 0.809447,
+      "epoch": 0.02374915754677621,
+      "grad_norm": 0.16049319505691528,
+      "learning_rate": 9.509956150664796e-05,
+      "loss": 0.6451148,
+      "memory(GiB)": 12.9,
+      "step": 185,
+      "train_speed(iter/s)": 0.082533
+    },
+    {
+      "acc": 0.80372143,
+      "epoch": 0.024391026669662055,
+      "grad_norm": 0.19566483795642853,
+      "learning_rate": 9.473646649103818e-05,
+      "loss": 0.65715637,
+      "memory(GiB)": 12.9,
+      "step": 190,
+      "train_speed(iter/s)": 0.083203
+    },
+    {
+      "acc": 0.81516018,
+      "epoch": 0.0250328957925479,
+      "grad_norm": 0.1836969554424286,
+      "learning_rate": 9.43611409721806e-05,
+      "loss": 0.63388405,
+      "memory(GiB)": 12.9,
+      "step": 195,
+      "train_speed(iter/s)": 0.083851
+    },
+    {
+      "acc": 0.80683355,
+      "epoch": 0.025674764915433742,
+      "grad_norm": 0.15840834379196167,
+      "learning_rate": 9.397368756032445e-05,
+      "loss": 0.65765018,
+      "memory(GiB)": 12.9,
+      "step": 200,
+      "train_speed(iter/s)": 0.084362
+    },
+    {
+      "epoch": 0.025674764915433742,
+      "eval_acc": 0.8154263931167957,
+      "eval_loss": 0.6316511034965515,
+      "eval_runtime": 194.3299,
+      "eval_samples_per_second": 6.474,
+      "eval_steps_per_second": 6.474,
+      "step": 200
+    },
+    {
+      "acc": 0.81476297,
+      "epoch": 0.026316634038319586,
+      "grad_norm": 0.18683095276355743,
+      "learning_rate": 9.357421218136386e-05,
+      "loss": 0.64739652,
+      "memory(GiB)": 12.9,
+      "step": 205,
+      "train_speed(iter/s)": 0.078515
+    },
+    {
+      "acc": 0.80957623,
+      "epoch": 0.02695850316120543,
+      "grad_norm": 0.16226844489574432,
+      "learning_rate": 9.316282404787871e-05,
+      "loss": 0.65772038,
+      "memory(GiB)": 12.9,
+      "step": 210,
+      "train_speed(iter/s)": 0.079027
+    },
+    {
+      "acc": 0.81580648,
+      "epoch": 0.027600372284091273,
+      "grad_norm": 0.16426005959510803,
+      "learning_rate": 9.273963562927695e-05,
+      "loss": 0.63748703,
+      "memory(GiB)": 12.9,
+      "step": 215,
+      "train_speed(iter/s)": 0.079584
+    },
+    {
+      "acc": 0.80812073,
+      "epoch": 0.028242241406977116,
+      "grad_norm": 0.1749752312898636,
+      "learning_rate": 9.230476262104677e-05,
+      "loss": 0.66415677,
+      "memory(GiB)": 12.9,
+      "step": 220,
+      "train_speed(iter/s)": 0.080111
+    },
+    {
+      "acc": 0.81957884,
+      "epoch": 0.02888411052986296,
+      "grad_norm": 0.15272143483161926,
+      "learning_rate": 9.185832391312644e-05,
+      "loss": 0.62169385,
+      "memory(GiB)": 12.9,
+      "step": 225,
+      "train_speed(iter/s)": 0.080603
+    },
+    {
+      "acc": 0.80860453,
+      "epoch": 0.029525979652748804,
+      "grad_norm": 0.1689305603504181,
+      "learning_rate": 9.140044155740101e-05,
+      "loss": 0.6528583,
+      "memory(GiB)": 12.9,
+      "step": 230,
+      "train_speed(iter/s)": 0.081113
+    },
+    {
+      "acc": 0.81558657,
+      "epoch": 0.030167848775634647,
+      "grad_norm": 0.20389845967292786,
+      "learning_rate": 9.093124073433463e-05,
+      "loss": 0.62348075,
+      "memory(GiB)": 12.9,
+      "step": 235,
+      "train_speed(iter/s)": 0.081633
+    },
+    {
+      "acc": 0.83771782,
+      "epoch": 0.03080971789852049,
+      "grad_norm": 0.1833639293909073,
+      "learning_rate": 9.045084971874738e-05,
+      "loss": 0.54310794,
+      "memory(GiB)": 12.9,
+      "step": 240,
+      "train_speed(iter/s)": 0.082163
+    },
+    {
+      "acc": 0.81091404,
+      "epoch": 0.03145158702140634,
+      "grad_norm": 0.1823006570339203,
+      "learning_rate": 8.995939984474624e-05,
+      "loss": 0.65011134,
+      "memory(GiB)": 12.9,
+      "step": 245,
+      "train_speed(iter/s)": 0.082662
+    },
+    {
+      "acc": 0.8202961,
+      "epoch": 0.03209345614429218,
+      "grad_norm": 0.19047267735004425,
+      "learning_rate": 8.945702546981969e-05,
+      "loss": 0.61326828,
+      "memory(GiB)": 12.9,
+      "step": 250,
+      "train_speed(iter/s)": 0.083111
+    },
+    {
+      "epoch": 0.03209345614429218,
+      "eval_acc": 0.8174640226702884,
+      "eval_loss": 0.6222989559173584,
+      "eval_runtime": 194.2619,
+      "eval_samples_per_second": 6.476,
+      "eval_steps_per_second": 6.476,
+      "step": 250
+    },
+    {
+      "acc": 0.81616802,
+      "epoch": 0.032735325267178025,
+      "grad_norm": 0.19024300575256348,
+      "learning_rate": 8.894386393810563e-05,
+      "loss": 0.65217943,
+      "memory(GiB)": 12.9,
+      "step": 255,
+      "train_speed(iter/s)": 0.078534
+    },
+    {
+      "acc": 0.81027069,
+      "epoch": 0.03337719439006387,
+      "grad_norm": 0.19169311225414276,
+      "learning_rate": 8.842005554284296e-05,
+      "loss": 0.65409112,
+      "memory(GiB)": 12.9,
+      "step": 260,
+      "train_speed(iter/s)": 0.078979
+    },
+    {
+      "acc": 0.80319004,
+      "epoch": 0.03401906351294971,
+      "grad_norm": 0.20081347227096558,
+      "learning_rate": 8.788574348801675e-05,
+      "loss": 0.67106547,
+      "memory(GiB)": 12.9,
+      "step": 265,
+      "train_speed(iter/s)": 0.079517
+    },
+    {
+      "acc": 0.82499094,
+      "epoch": 0.034660932635835556,
+      "grad_norm": 0.16266761720180511,
+      "learning_rate": 8.73410738492077e-05,
+      "loss": 0.57896609,
+      "memory(GiB)": 12.9,
+      "step": 270,
+      "train_speed(iter/s)": 0.079959
+    },
+    {
+      "acc": 0.83434772,
+      "epoch": 0.0353028017587214,
+      "grad_norm": 0.19760368764400482,
+      "learning_rate": 8.678619553365659e-05,
+      "loss": 0.56308613,
+      "memory(GiB)": 12.9,
+      "step": 275,
+      "train_speed(iter/s)": 0.080441
+    },
+    {
+      "acc": 0.81103878,
+      "epoch": 0.035944670881607244,
+      "grad_norm": 0.20421290397644043,
+      "learning_rate": 8.622126023955446e-05,
+      "loss": 0.62751951,
+      "memory(GiB)": 12.9,
+      "step": 280,
+      "train_speed(iter/s)": 0.080888
+    },
+    {
+      "acc": 0.80577688,
+      "epoch": 0.03658654000449309,
+      "grad_norm": 0.18944613635540009,
+      "learning_rate": 8.564642241456986e-05,
+      "loss": 0.67880988,
+      "memory(GiB)": 12.9,
+      "step": 285,
+      "train_speed(iter/s)": 0.081351
+    },
+    {
+      "acc": 0.81629257,
+      "epoch": 0.03722840912737893,
+      "grad_norm": 0.16580034792423248,
+      "learning_rate": 8.506183921362443e-05,
+      "loss": 0.63270054,
+      "memory(GiB)": 12.9,
+      "step": 290,
+      "train_speed(iter/s)": 0.081754
+    },
+    {
+      "acc": 0.81265612,
+      "epoch": 0.037870278250264774,
+      "grad_norm": 0.20476555824279785,
+      "learning_rate": 8.44676704559283e-05,
+      "loss": 0.63396297,
+      "memory(GiB)": 12.9,
+      "step": 295,
+      "train_speed(iter/s)": 0.082163
+    },
+    {
+      "acc": 0.81665688,
+      "epoch": 0.03851214737315061,
+      "grad_norm": 0.1685812920331955,
+      "learning_rate": 8.386407858128706e-05,
+      "loss": 0.63431234,
+      "memory(GiB)": 12.9,
+      "step": 300,
+      "train_speed(iter/s)": 0.082563
+    },
+    {
+      "epoch": 0.03851214737315061,
+      "eval_acc": 0.8180910416791721,
+      "eval_loss": 0.6213402152061462,
+      "eval_runtime": 194.9277,
+      "eval_samples_per_second": 6.454,
+      "eval_steps_per_second": 6.454,
+      "step": 300
+    },
+    {
+      "acc": 0.81313,
+      "epoch": 0.039154016496036455,
+      "grad_norm": 0.19343021512031555,
+      "learning_rate": 8.32512286056924e-05,
+      "loss": 0.65134788,
+      "memory(GiB)": 12.9,
+      "step": 305,
+      "train_speed(iter/s)": 0.078839
+    },
+    {
+      "acc": 0.80900688,
+      "epoch": 0.0397958856189223,
+      "grad_norm": 0.20626357197761536,
+      "learning_rate": 8.262928807620843e-05,
+      "loss": 0.66552057,
+      "memory(GiB)": 12.9,
+      "step": 310,
+      "train_speed(iter/s)": 0.07924
+    },
+    {
+      "acc": 0.81944618,
+      "epoch": 0.04043775474180814,
+      "grad_norm": 0.22261416912078857,
+      "learning_rate": 8.199842702516583e-05,
+      "loss": 0.6285687,
+      "memory(GiB)": 12.9,
+      "step": 315,
+      "train_speed(iter/s)": 0.079651
+    },
+    {
+      "acc": 0.82105675,
+      "epoch": 0.041079623864693986,
+      "grad_norm": 0.17977821826934814,
+      "learning_rate": 8.135881792367686e-05,
+      "loss": 0.61407933,
+      "memory(GiB)": 12.9,
+      "step": 320,
+      "train_speed(iter/s)": 0.080053
+    },
+    {
+      "acc": 0.81391029,
+      "epoch": 0.04172149298757983,
+      "grad_norm": 0.1734636276960373,
+      "learning_rate": 8.07106356344834e-05,
+      "loss": 0.63154807,
+      "memory(GiB)": 12.9,
+      "step": 325,
+      "train_speed(iter/s)": 0.080427
+    },
+    {
+      "acc": 0.80443487,
+      "epoch": 0.04236336211046567,
+      "grad_norm": 0.20138931274414062,
+      "learning_rate": 8.005405736415126e-05,
+      "loss": 0.67294674,
+      "memory(GiB)": 12.9,
+      "step": 330,
+      "train_speed(iter/s)": 0.08081
+    },
+    {
+      "acc": 0.8138341,
+      "epoch": 0.04300523123335152,
+      "grad_norm": 0.2296956181526184,
+      "learning_rate": 7.938926261462366e-05,
+      "loss": 0.62769508,
+      "memory(GiB)": 12.9,
+      "step": 335,
+      "train_speed(iter/s)": 0.081179
+    },
+    {
+      "acc": 0.81060715,
+      "epoch": 0.04364710035623736,
+      "grad_norm": 0.18877319991588593,
+      "learning_rate": 7.871643313414718e-05,
+      "loss": 0.65062008,
+      "memory(GiB)": 12.9,
+      "step": 340,
+      "train_speed(iter/s)": 0.081555
+    },
+    {
+      "acc": 0.80925789,
+      "epoch": 0.044288969479123204,
+      "grad_norm": 0.20824845135211945,
+      "learning_rate": 7.803575286758364e-05,
+      "loss": 0.65066905,
+      "memory(GiB)": 12.9,
+      "step": 345,
+      "train_speed(iter/s)": 0.081899
+    },
+    {
+      "acc": 0.80743494,
+      "epoch": 0.04493083860200905,
+      "grad_norm": 0.1929645985364914,
+      "learning_rate": 7.734740790612136e-05,
+      "loss": 0.64833279,
+      "memory(GiB)": 12.9,
+      "step": 350,
+      "train_speed(iter/s)": 0.082229
+    },
+    {
+      "epoch": 0.04493083860200905,
+      "eval_acc": 0.818597113383227,
+      "eval_loss": 0.6184135675430298,
+      "eval_runtime": 195.2572,
+      "eval_samples_per_second": 6.443,
+      "eval_steps_per_second": 6.443,
+      "step": 350
+    },
+    {
+      "acc": 0.81946774,
+      "epoch": 0.04557270772489489,
+      "grad_norm": 0.185298889875412,
+      "learning_rate": 7.66515864363997e-05,
+      "loss": 0.63578596,
+      "memory(GiB)": 12.9,
+      "step": 355,
+      "train_speed(iter/s)": 0.078909
+    },
+    {
+      "acc": 0.82359991,
+      "epoch": 0.046214576847780735,
+      "grad_norm": 0.16975072026252747,
+      "learning_rate": 7.594847868906076e-05,
+      "loss": 0.59197049,
+      "memory(GiB)": 12.9,
+      "step": 360,
+      "train_speed(iter/s)": 0.079228
+    },
+    {
+      "acc": 0.82826347,
+      "epoch": 0.04685644597066658,
+      "grad_norm": 0.19332191348075867,
+      "learning_rate": 7.52382768867422e-05,
+      "loss": 0.58269887,
+      "memory(GiB)": 12.9,
+      "step": 365,
+      "train_speed(iter/s)": 0.079527
+    },
+    {
+      "acc": 0.81692057,
+      "epoch": 0.04749831509355242,
+      "grad_norm": 0.1801847517490387,
+      "learning_rate": 7.452117519152542e-05,
+      "loss": 0.62563605,
+      "memory(GiB)": 12.9,
+      "step": 370,
+      "train_speed(iter/s)": 0.079855
+    },
+    {
+      "acc": 0.81458244,
+      "epoch": 0.048140184216438266,
+      "grad_norm": 0.2671217620372772,
+      "learning_rate": 7.379736965185368e-05,
+      "loss": 0.62537222,
+      "memory(GiB)": 12.9,
+      "step": 375,
+      "train_speed(iter/s)": 0.080167
+    },
+    {
+      "acc": 0.80892944,
+      "epoch": 0.04878205333932411,
+      "grad_norm": 0.20377114415168762,
+      "learning_rate": 7.30670581489344e-05,
+      "loss": 0.6464529,
+      "memory(GiB)": 12.9,
+      "step": 380,
+      "train_speed(iter/s)": 0.080535
+    },
+    {
+      "acc": 0.81387253,
+      "epoch": 0.04942392246220995,
+      "grad_norm": 0.21851402521133423,
+      "learning_rate": 7.233044034264034e-05,
+      "loss": 0.64175944,
+      "memory(GiB)": 12.9,
+      "step": 385,
+      "train_speed(iter/s)": 0.080866
+    },
+    {
+      "acc": 0.81104851,
+      "epoch": 0.0500657915850958,
+      "grad_norm": 0.17920850217342377,
+      "learning_rate": 7.158771761692464e-05,
+      "loss": 0.65399485,
+      "memory(GiB)": 12.9,
+      "step": 390,
+      "train_speed(iter/s)": 0.081177
+    },
+    {
+      "acc": 0.81167917,
+      "epoch": 0.05070766070798164,
+      "grad_norm": 0.21850617229938507,
+      "learning_rate": 7.083909302476453e-05,
+      "loss": 0.65062542,
+      "memory(GiB)": 12.9,
+      "step": 395,
+      "train_speed(iter/s)": 0.081491
+    },
+    {
+      "acc": 0.82351618,
+      "epoch": 0.051349529830867484,
+      "grad_norm": 0.1936645656824112,
+      "learning_rate": 7.008477123264848e-05,
+      "loss": 0.62041502,
+      "memory(GiB)": 12.9,
+      "step": 400,
+      "train_speed(iter/s)": 0.081765
+    },
+    {
+      "epoch": 0.051349529830867484,
+      "eval_acc": 0.8192219173956314,
+      "eval_loss": 0.6147059202194214,
+      "eval_runtime": 194.2328,
+      "eval_samples_per_second": 6.477,
+      "eval_steps_per_second": 6.477,
+      "step": 400
+    },
+    {
+      "acc": 0.81023674,
+      "epoch": 0.05199139895375333,
+      "grad_norm": 0.1859249472618103,
+      "learning_rate": 6.932495846462261e-05,
+      "loss": 0.63979912,
+      "memory(GiB)": 12.9,
+      "step": 405,
+      "train_speed(iter/s)": 0.078921
+    },
+    {
+      "acc": 0.81422606,
+      "epoch": 0.05263326807663917,
+      "grad_norm": 0.18879809975624084,
+      "learning_rate": 6.855986244591104e-05,
+      "loss": 0.63082585,
+      "memory(GiB)": 12.9,
+      "step": 410,
+      "train_speed(iter/s)": 0.079225
+    },
+    {
+      "acc": 0.82355537,
+      "epoch": 0.053275137199525015,
+      "grad_norm": 0.1991114318370819,
+      "learning_rate": 6.778969234612584e-05,
+      "loss": 0.60244231,
+      "memory(GiB)": 12.9,
+      "step": 415,
+      "train_speed(iter/s)": 0.079522
+    },
+    {
+      "acc": 0.80394392,
+      "epoch": 0.05391700632241086,
+      "grad_norm": 0.18245312571525574,
+      "learning_rate": 6.701465872208216e-05,
+      "loss": 0.68041868,
+      "memory(GiB)": 12.9,
+      "step": 420,
+      "train_speed(iter/s)": 0.079822
+    },
+    {
+      "acc": 0.82535486,
+      "epoch": 0.0545588754452967,
+      "grad_norm": 0.18471555411815643,
+      "learning_rate": 6.623497346023418e-05,
+      "loss": 0.58148174,
+      "memory(GiB)": 12.9,
+      "step": 425,
+      "train_speed(iter/s)": 0.080165
+    },
+    {
+      "acc": 0.8145462,
+      "epoch": 0.055200744568182546,
+      "grad_norm": 0.21886418759822845,
+      "learning_rate": 6.545084971874738e-05,
+      "loss": 0.64324603,
+      "memory(GiB)": 12.9,
+      "step": 430,
+      "train_speed(iter/s)": 0.080479
+    },
+    {
+      "acc": 0.80875597,
+      "epoch": 0.05584261369106839,
+      "grad_norm": 0.2046084851026535,
+      "learning_rate": 6.466250186922325e-05,
+      "loss": 0.64245248,
+      "memory(GiB)": 12.9,
+      "step": 435,
+      "train_speed(iter/s)": 0.080772
+    },
+    {
+      "acc": 0.81591587,
+      "epoch": 0.05648448281395423,
+      "grad_norm": 0.1934758424758911,
+      "learning_rate": 6.387014543809223e-05,
+      "loss": 0.63060322,
+      "memory(GiB)": 12.9,
+      "step": 440,
+      "train_speed(iter/s)": 0.081046
+    },
+    {
+      "acc": 0.81191225,
+      "epoch": 0.05712635193684008,
+      "grad_norm": 0.2059066742658615,
+      "learning_rate": 6.307399704769099e-05,
+      "loss": 0.64218235,
+      "memory(GiB)": 12.9,
+      "step": 445,
+      "train_speed(iter/s)": 0.081334
+    },
+    {
+      "acc": 0.81284475,
+      "epoch": 0.05776822105972592,
+      "grad_norm": 0.18999861180782318,
+      "learning_rate": 6.227427435703997e-05,
+      "loss": 0.64007626,
+      "memory(GiB)": 12.9,
+      "step": 450,
+      "train_speed(iter/s)": 0.081619
+    },
+    {
+      "epoch": 0.05776822105972592,
+      "eval_acc": 0.8195972020245513,
+      "eval_loss": 0.6134685277938843,
+      "eval_runtime": 194.9348,
+      "eval_samples_per_second": 6.453,
+      "eval_steps_per_second": 6.453,
+      "step": 450
+    },
+    {
+      "acc": 0.81439934,
+      "epoch": 0.058410090182611764,
+      "grad_norm": 0.20194242894649506,
+      "learning_rate": 6.147119600233758e-05,
+      "loss": 0.63169065,
+      "memory(GiB)": 12.9,
+      "step": 455,
+      "train_speed(iter/s)": 0.07909
+    },
+    {
+      "acc": 0.80925694,
+      "epoch": 0.05905195930549761,
+      "grad_norm": 0.22199371457099915,
+      "learning_rate": 6.066498153718735e-05,
+      "loss": 0.65331206,
+      "memory(GiB)": 12.9,
+      "step": 460,
+      "train_speed(iter/s)": 0.079394
+    },
+    {
+      "acc": 0.81743002,
+      "epoch": 0.05969382842838345,
+      "grad_norm": 0.1933964639902115,
+      "learning_rate": 5.985585137257401e-05,
+      "loss": 0.62136326,
+      "memory(GiB)": 12.9,
+      "step": 465,
+      "train_speed(iter/s)": 0.079664
+    },
+    {
+      "acc": 0.82081652,
+      "epoch": 0.060335697551269295,
+      "grad_norm": 0.19113869965076447,
+      "learning_rate": 5.90440267166055e-05,
+      "loss": 0.58844328,
+      "memory(GiB)": 12.9,
+      "step": 470,
+      "train_speed(iter/s)": 0.079903
+    },
+    {
+      "acc": 0.82189302,
+      "epoch": 0.06097756667415514,
+      "grad_norm": 0.21947172284126282,
+      "learning_rate": 5.8229729514036705e-05,
+      "loss": 0.6088398,
+      "memory(GiB)": 12.9,
+      "step": 475,
+      "train_speed(iter/s)": 0.080178
+    },
+    {
+      "acc": 0.81938705,
+      "epoch": 0.06161943579704098,
+      "grad_norm": 0.19398923218250275,
+      "learning_rate": 5.74131823855921e-05,
+      "loss": 0.62442837,
+      "memory(GiB)": 12.9,
+      "step": 480,
+      "train_speed(iter/s)": 0.080433
+    },
+    {
+      "acc": 0.82784338,
+      "epoch": 0.062261304919926826,
+      "grad_norm": 0.17412325739860535,
+      "learning_rate": 5.6594608567103456e-05,
+      "loss": 0.57895641,
+      "memory(GiB)": 12.9,
+      "step": 485,
+      "train_speed(iter/s)": 0.080668
+    },
+    {
+      "acc": 0.82182274,
+      "epoch": 0.06290317404281268,
+      "grad_norm": 0.2373647540807724,
+      "learning_rate": 5.577423184847932e-05,
+      "loss": 0.61787653,
+      "memory(GiB)": 12.9,
+      "step": 490,
+      "train_speed(iter/s)": 0.080927
+    },
+    {
+      "acc": 0.82861795,
+      "epoch": 0.06354504316569852,
+      "grad_norm": 0.18874205648899078,
+      "learning_rate": 5.495227651252315e-05,
+      "loss": 0.57188206,
+      "memory(GiB)": 12.9,
+      "step": 495,
+      "train_speed(iter/s)": 0.081151
+    },
+    {
+      "acc": 0.80789671,
+      "epoch": 0.06418691228858436,
+      "grad_norm": 0.1971614509820938,
+      "learning_rate": 5.4128967273616625e-05,
+      "loss": 0.67568989,
+      "memory(GiB)": 12.9,
+      "step": 500,
+      "train_speed(iter/s)": 0.081405
+    },
+    {
+      "epoch": 0.06418691228858436,
+      "eval_acc": 0.8191289863904123,
+      "eval_loss": 0.6154261827468872,
+      "eval_runtime": 194.8211,
+      "eval_samples_per_second": 6.457,
+      "eval_steps_per_second": 6.457,
+      "step": 500
+    },
+    {
+      "acc": 0.80737734,
+      "epoch": 0.06482878141147021,
+      "grad_norm": 0.2034166306257248,
+      "learning_rate": 5.330452921628497e-05,
+      "loss": 0.64408102,
+      "memory(GiB)": 12.9,
+      "step": 505,
+      "train_speed(iter/s)": 0.079157
+    },
+    {
+      "acc": 0.83580818,
+      "epoch": 0.06547065053435605,
+      "grad_norm": 0.1943274289369583,
+      "learning_rate": 5.247918773366112e-05,
+      "loss": 0.55616164,
+      "memory(GiB)": 12.9,
+      "step": 510,
+      "train_speed(iter/s)": 0.079418
+    },
+    {
+      "acc": 0.81569843,
+      "epoch": 0.0661125196572419,
+      "grad_norm": 0.21546119451522827,
+      "learning_rate": 5.165316846586541e-05,
+      "loss": 0.63107128,
+      "memory(GiB)": 12.9,
+      "step": 515,
+      "train_speed(iter/s)": 0.079676
+    },
+    {
+      "acc": 0.82521887,
+      "epoch": 0.06675438878012774,
+      "grad_norm": 0.18064647912979126,
+      "learning_rate": 5.0826697238317935e-05,
+      "loss": 0.60510926,
+      "memory(GiB)": 12.9,
+      "step": 520,
+      "train_speed(iter/s)": 0.079928
+    },
+    {
+      "acc": 0.8060174,
+      "epoch": 0.06739625790301358,
+      "grad_norm": 0.2181115299463272,
+      "learning_rate": 5e-05,
+      "loss": 0.65878468,
+      "memory(GiB)": 12.9,
+      "step": 525,
+      "train_speed(iter/s)": 0.080188
+    },
+    {
+      "acc": 0.81847181,
+      "epoch": 0.06803812702589943,
+      "grad_norm": 0.19227808713912964,
+      "learning_rate": 4.917330276168208e-05,
+      "loss": 0.61963496,
+      "memory(GiB)": 12.9,
+      "step": 530,
+      "train_speed(iter/s)": 0.080404
+    },
+    {
+      "acc": 0.81498117,
+      "epoch": 0.06867999614878527,
+      "grad_norm": 0.22034384310245514,
+      "learning_rate": 4.834683153413459e-05,
+      "loss": 0.6185873,
+      "memory(GiB)": 12.9,
+      "step": 535,
+      "train_speed(iter/s)": 0.080637
+    },
+    {
+      "acc": 0.80154915,
+      "epoch": 0.06932186527167111,
+      "grad_norm": 0.19249100983142853,
+      "learning_rate": 4.7520812266338885e-05,
+      "loss": 0.6963346,
+      "memory(GiB)": 12.9,
+      "step": 540,
+      "train_speed(iter/s)": 0.080856
+    },
+    {
+      "acc": 0.81683502,
+      "epoch": 0.06996373439455696,
+      "grad_norm": 0.1994090974330902,
+      "learning_rate": 4.669547078371504e-05,
+      "loss": 0.6188817,
+      "memory(GiB)": 12.9,
+      "step": 545,
+      "train_speed(iter/s)": 0.081075
+    },
+    {
+      "acc": 0.82736206,
+      "epoch": 0.0706056035174428,
+      "grad_norm": 0.18659113347530365,
+      "learning_rate": 4.5871032726383386e-05,
+      "loss": 0.59354177,
+      "memory(GiB)": 12.9,
+      "step": 550,
+      "train_speed(iter/s)": 0.081266
+    },
+    {
+      "epoch": 0.0706056035174428,
+      "eval_acc": 0.8192911509362449,
+      "eval_loss": 0.6139300465583801,
+      "eval_runtime": 200.9073,
+      "eval_samples_per_second": 6.262,
+      "eval_steps_per_second": 6.262,
+      "step": 550
+    },
+    {
+      "acc": 0.81306267,
+      "epoch": 0.07124747264032864,
+      "grad_norm": 0.17537347972393036,
+      "learning_rate": 4.504772348747687e-05,
+      "loss": 0.63172441,
+      "memory(GiB)": 12.9,
+      "step": 555,
+      "train_speed(iter/s)": 0.079145
+    },
+    {
+      "acc": 0.82534027,
+      "epoch": 0.07188934176321449,
+      "grad_norm": 0.21789848804473877,
+      "learning_rate": 4.4225768151520694e-05,
+      "loss": 0.58555121,
+      "memory(GiB)": 12.9,
+      "step": 560,
+      "train_speed(iter/s)": 0.079394
+    },
+    {
+      "acc": 0.82140818,
+      "epoch": 0.07253121088610033,
+      "grad_norm": 0.19700835645198822,
+      "learning_rate": 4.3405391432896555e-05,
+      "loss": 0.59516869,
+      "memory(GiB)": 12.9,
+      "step": 565,
+      "train_speed(iter/s)": 0.079597
+    },
+    {
+      "acc": 0.81845322,
+      "epoch": 0.07317308000898617,
+      "grad_norm": 0.1818612515926361,
+      "learning_rate": 4.2586817614407895e-05,
+      "loss": 0.60017176,
+      "memory(GiB)": 12.9,
+      "step": 570,
+      "train_speed(iter/s)": 0.079791
+    },
+    {
+      "acc": 0.82169409,
+      "epoch": 0.07381494913187202,
+      "grad_norm": 0.2091558277606964,
+      "learning_rate": 4.17702704859633e-05,
+      "loss": 0.59121842,
+      "memory(GiB)": 12.9,
+      "step": 575,
+      "train_speed(iter/s)": 0.080013
+    },
+    {
+      "acc": 0.81168118,
+      "epoch": 0.07445681825475786,
+      "grad_norm": 0.20149122178554535,
+      "learning_rate": 4.095597328339452e-05,
+      "loss": 0.64874387,
+      "memory(GiB)": 12.9,
+      "step": 580,
+      "train_speed(iter/s)": 0.080249
+    },
+    {
+      "acc": 0.82017975,
+      "epoch": 0.0750986873776437,
+      "grad_norm": 0.21395570039749146,
+      "learning_rate": 4.0144148627425993e-05,
+      "loss": 0.63141985,
+      "memory(GiB)": 12.9,
+      "step": 585,
+      "train_speed(iter/s)": 0.080445
+    },
+    {
+      "acc": 0.8225503,
+      "epoch": 0.07574055650052955,
+      "grad_norm": 0.21270450949668884,
+      "learning_rate": 3.933501846281267e-05,
+      "loss": 0.59338393,
+      "memory(GiB)": 12.9,
+      "step": 590,
+      "train_speed(iter/s)": 0.080661
+    },
+    {
+      "acc": 0.80733299,
+      "epoch": 0.07638242562341538,
+      "grad_norm": 0.21861939132213593,
+      "learning_rate": 3.852880399766243e-05,
+      "loss": 0.65440679,
+      "memory(GiB)": 12.9,
+      "step": 595,
+      "train_speed(iter/s)": 0.080898
+    },
+    {
+      "acc": 0.81755505,
+      "epoch": 0.07702429474630122,
+      "grad_norm": 0.1930130422115326,
+      "learning_rate": 3.772572564296005e-05,
+      "loss": 0.61422372,
+      "memory(GiB)": 12.9,
+      "step": 600,
+      "train_speed(iter/s)": 0.081091
+    },
+    {
+      "epoch": 0.07702429474630122,
+      "eval_acc": 0.8198276458076612,
+      "eval_loss": 0.6109773516654968,
+      "eval_runtime": 195.2232,
+      "eval_samples_per_second": 6.444,
+      "eval_steps_per_second": 6.444,
+      "step": 600
+    },
+    {
+      "acc": 0.82081652,
+      "epoch": 0.07766616386918707,
+      "grad_norm": 0.20753733813762665,
+      "learning_rate": 3.6926002952309016e-05,
+      "loss": 0.6057888,
+      "memory(GiB)": 12.9,
+      "step": 605,
+      "train_speed(iter/s)": 0.07921
+    },
+    {
+      "acc": 0.82343655,
+      "epoch": 0.07830803299207291,
+      "grad_norm": 0.1979513019323349,
+      "learning_rate": 3.612985456190778e-05,
+      "loss": 0.61725841,
+      "memory(GiB)": 12.9,
+      "step": 610,
+      "train_speed(iter/s)": 0.079401
+    },
+    {
+      "acc": 0.81410999,
+      "epoch": 0.07894990211495875,
+      "grad_norm": 0.20070388913154602,
+      "learning_rate": 3.533749813077677e-05,
+      "loss": 0.62781944,
+      "memory(GiB)": 12.9,
+      "step": 615,
+      "train_speed(iter/s)": 0.079569
+    },
+    {
+      "acc": 0.81817207,
+      "epoch": 0.0795917712378446,
+      "grad_norm": 0.2327299267053604,
+      "learning_rate": 3.4549150281252636e-05,
+      "loss": 0.62505813,
+      "memory(GiB)": 12.9,
+      "step": 620,
+      "train_speed(iter/s)": 0.079788
+    },
+    {
+      "acc": 0.82294455,
+      "epoch": 0.08023364036073044,
+      "grad_norm": 0.18113943934440613,
+      "learning_rate": 3.3765026539765834e-05,
+      "loss": 0.60028605,
+      "memory(GiB)": 12.9,
+      "step": 625,
+      "train_speed(iter/s)": 0.079975
+    },
+    {
+      "acc": 0.82047234,
+      "epoch": 0.08087550948361628,
+      "grad_norm": 0.20408566296100616,
+      "learning_rate": 3.298534127791785e-05,
+      "loss": 0.61079011,
+      "memory(GiB)": 12.9,
+      "step": 630,
+      "train_speed(iter/s)": 0.080162
+    },
+    {
+      "acc": 0.82677517,
+      "epoch": 0.08151737860650213,
+      "grad_norm": 0.2132798135280609,
+      "learning_rate": 3.221030765387417e-05,
+      "loss": 0.57911015,
+      "memory(GiB)": 12.9,
+      "step": 635,
+      "train_speed(iter/s)": 0.08036
+    },
+    {
+      "acc": 0.81979742,
+      "epoch": 0.08215924772938797,
+      "grad_norm": 0.20325441658496857,
+      "learning_rate": 3.144013755408895e-05,
+      "loss": 0.60286283,
+      "memory(GiB)": 12.9,
+      "step": 640,
+      "train_speed(iter/s)": 0.080558
+    },
+    {
+      "acc": 0.83071327,
+      "epoch": 0.08280111685227382,
+      "grad_norm": 0.20148617029190063,
+      "learning_rate": 3.0675041535377405e-05,
+      "loss": 0.58266845,
+      "memory(GiB)": 12.9,
+      "step": 645,
+      "train_speed(iter/s)": 0.080764
+    },
+    {
+      "acc": 0.83186378,
+      "epoch": 0.08344298597515966,
+      "grad_norm": 0.19807127118110657,
+      "learning_rate": 2.991522876735154e-05,
+      "loss": 0.55458455,
+      "memory(GiB)": 12.9,
+      "step": 650,
+      "train_speed(iter/s)": 0.080948
+    },
+    {
+      "epoch": 0.08344298597515966,
+      "eval_acc": 0.8209896385738762,
+      "eval_loss": 0.6076882481575012,
+      "eval_runtime": 194.4544,
+      "eval_samples_per_second": 6.469,
+      "eval_steps_per_second": 6.469,
+      "step": 650
+    },
+    {
+      "acc": 0.82214499,
+      "epoch": 0.0840848550980455,
+      "grad_norm": 0.18806923925876617,
+      "learning_rate": 2.916090697523549e-05,
+      "loss": 0.61260557,
+      "memory(GiB)": 12.9,
+      "step": 655,
+      "train_speed(iter/s)": 0.079195
+    },
+    {
+      "acc": 0.82406235,
+      "epoch": 0.08472672422093135,
+      "grad_norm": 0.19712701439857483,
+      "learning_rate": 2.8412282383075363e-05,
+      "loss": 0.59967513,
+      "memory(GiB)": 12.9,
+      "step": 660,
+      "train_speed(iter/s)": 0.079403
+    },
+    {
+      "acc": 0.80978308,
+      "epoch": 0.08536859334381719,
+      "grad_norm": 0.19845040142536163,
+      "learning_rate": 2.766955965735968e-05,
+      "loss": 0.64144659,
+      "memory(GiB)": 12.9,
+      "step": 665,
+      "train_speed(iter/s)": 0.079587
+    },
+    {
+      "acc": 0.81445723,
+      "epoch": 0.08601046246670303,
+      "grad_norm": 0.21892952919006348,
+      "learning_rate": 2.693294185106562e-05,
+      "loss": 0.62947636,
+      "memory(GiB)": 12.9,
+      "step": 670,
+      "train_speed(iter/s)": 0.079776
+    },
+    {
+      "acc": 0.82742786,
+      "epoch": 0.08665233158958888,
+      "grad_norm": 0.21680107712745667,
+      "learning_rate": 2.6202630348146324e-05,
+      "loss": 0.58508019,
+      "memory(GiB)": 12.9,
+      "step": 675,
+      "train_speed(iter/s)": 0.07997
+    },
+    {
+      "acc": 0.82206249,
+      "epoch": 0.08729420071247472,
+      "grad_norm": 0.19368013739585876,
+      "learning_rate": 2.547882480847461e-05,
+      "loss": 0.60457458,
+      "memory(GiB)": 12.9,
+      "step": 680,
+      "train_speed(iter/s)": 0.080131
+    },
+    {
+      "acc": 0.8187645,
+      "epoch": 0.08793606983536056,
+      "grad_norm": 0.21237029135227203,
+      "learning_rate": 2.476172311325783e-05,
+      "loss": 0.61911306,
+      "memory(GiB)": 12.9,
+      "step": 685,
+      "train_speed(iter/s)": 0.080301
+    },
+    {
+      "acc": 0.81704865,
+      "epoch": 0.08857793895824641,
+      "grad_norm": 0.22553302347660065,
+      "learning_rate": 2.405152131093926e-05,
+      "loss": 0.63167734,
+      "memory(GiB)": 12.9,
+      "step": 690,
+      "train_speed(iter/s)": 0.080481
+    },
+    {
+      "acc": 0.81163187,
+      "epoch": 0.08921980808113225,
+      "grad_norm": 0.21230626106262207,
+      "learning_rate": 2.3348413563600325e-05,
+      "loss": 0.63321238,
+      "memory(GiB)": 12.9,
+      "step": 695,
+      "train_speed(iter/s)": 0.080651
+    },
+    {
+      "acc": 0.81947212,
+      "epoch": 0.0898616772040181,
+      "grad_norm": 0.23210006952285767,
+      "learning_rate": 2.2652592093878666e-05,
+      "loss": 0.61235924,
+      "memory(GiB)": 12.9,
+      "step": 700,
+      "train_speed(iter/s)": 0.08085
+    },
+    {
+      "epoch": 0.0898616772040181,
+      "eval_acc": 0.8203855599214145,
+      "eval_loss": 0.6076461672782898,
+      "eval_runtime": 194.4087,
+      "eval_samples_per_second": 6.471,
+      "eval_steps_per_second": 6.471,
+      "step": 700
+    },
+    {
+      "acc": 0.81527042,
+      "epoch": 0.09050354632690394,
+      "grad_norm": 0.20815226435661316,
+      "learning_rate": 2.196424713241637e-05,
+      "loss": 0.63990374,
+      "memory(GiB)": 12.9,
+      "step": 705,
+      "train_speed(iter/s)": 0.079227
+    },
+    {
+      "acc": 0.82444172,
+      "epoch": 0.09114541544978978,
+      "grad_norm": 0.21324452757835388,
+      "learning_rate": 2.128356686585282e-05,
+      "loss": 0.59049015,
+      "memory(GiB)": 12.9,
+      "step": 710,
+      "train_speed(iter/s)": 0.079404
+    },
+    {
+      "acc": 0.81617746,
+      "epoch": 0.09178728457267563,
+      "grad_norm": 0.19047819077968597,
+      "learning_rate": 2.061073738537635e-05,
+      "loss": 0.60989962,
+      "memory(GiB)": 12.9,
+      "step": 715,
+      "train_speed(iter/s)": 0.079576
+    },
+    {
+      "acc": 0.80728264,
+      "epoch": 0.09242915369556147,
+      "grad_norm": 0.2062956988811493,
+      "learning_rate": 1.9945942635848748e-05,
+      "loss": 0.67080445,
+      "memory(GiB)": 12.9,
+      "step": 720,
+      "train_speed(iter/s)": 0.079742
+    },
+    {
+      "acc": 0.81671677,
+      "epoch": 0.09307102281844731,
+      "grad_norm": 0.19861043989658356,
+      "learning_rate": 1.928936436551661e-05,
+      "loss": 0.61908646,
+      "memory(GiB)": 12.9,
+      "step": 725,
+      "train_speed(iter/s)": 0.079905
+    },
+    {
+      "acc": 0.81090012,
+      "epoch": 0.09371289194133316,
+      "grad_norm": 0.19133475422859192,
+      "learning_rate": 1.8641182076323148e-05,
+      "loss": 0.66370716,
+      "memory(GiB)": 12.9,
+      "step": 730,
+      "train_speed(iter/s)": 0.08006
+    },
+    {
+      "acc": 0.81192856,
+      "epoch": 0.094354761064219,
+      "grad_norm": 0.22429423034191132,
+      "learning_rate": 1.800157297483417e-05,
+      "loss": 0.66476107,
+      "memory(GiB)": 12.9,
+      "step": 735,
+      "train_speed(iter/s)": 0.080239
+    },
+    {
+      "acc": 0.82513466,
+      "epoch": 0.09499663018710484,
+      "grad_norm": 0.21653637290000916,
+      "learning_rate": 1.7370711923791567e-05,
+      "loss": 0.58692684,
+      "memory(GiB)": 12.9,
+      "step": 740,
+      "train_speed(iter/s)": 0.080402
+    },
+    {
+      "acc": 0.82555256,
+      "epoch": 0.09563849930999069,
+      "grad_norm": 0.17780368030071259,
+      "learning_rate": 1.6748771394307585e-05,
+      "loss": 0.57462683,
+      "memory(GiB)": 12.9,
+      "step": 745,
+      "train_speed(iter/s)": 0.080553
+    },
+    {
+      "acc": 0.81813164,
+      "epoch": 0.09628036843287653,
+      "grad_norm": 0.2015465795993805,
+      "learning_rate": 1.6135921418712956e-05,
+      "loss": 0.61962047,
+      "memory(GiB)": 12.9,
+      "step": 750,
+      "train_speed(iter/s)": 0.080707
+    },
+    {
+      "epoch": 0.09628036843287653,
+      "eval_acc": 0.8204207990619473,
+      "eval_loss": 0.6083427667617798,
+      "eval_runtime": 194.7163,
+      "eval_samples_per_second": 6.461,
+      "eval_steps_per_second": 6.461,
+      "step": 750
+    },
+    {
+      "acc": 0.82061501,
+      "epoch": 0.09692223755576237,
+      "grad_norm": 0.2477208524942398,
+      "learning_rate": 1.553232954407171e-05,
+      "loss": 0.61750922,
+      "memory(GiB)": 12.9,
+      "step": 755,
+      "train_speed(iter/s)": 0.079226
+    },
+    {
+      "acc": 0.81846237,
+      "epoch": 0.09756410667864822,
+      "grad_norm": 0.22278401255607605,
+      "learning_rate": 1.4938160786375572e-05,
+      "loss": 0.61280818,
+      "memory(GiB)": 12.9,
+      "step": 760,
+      "train_speed(iter/s)": 0.079391
+    },
+    {
+      "acc": 0.81434851,
+      "epoch": 0.09820597580153406,
+      "grad_norm": 0.22640761733055115,
+      "learning_rate": 1.435357758543015e-05,
+      "loss": 0.63006077,
+      "memory(GiB)": 12.9,
+      "step": 765,
+      "train_speed(iter/s)": 0.079552
+    },
+    {
+      "acc": 0.81086941,
+      "epoch": 0.0988478449244199,
+      "grad_norm": 0.20250476896762848,
+      "learning_rate": 1.3778739760445552e-05,
+      "loss": 0.63865561,
+      "memory(GiB)": 12.9,
+      "step": 770,
+      "train_speed(iter/s)": 0.079713
+    },
+    {
+      "acc": 0.81333466,
+      "epoch": 0.09948971404730575,
+      "grad_norm": 0.21983562409877777,
+      "learning_rate": 1.3213804466343421e-05,
+      "loss": 0.63764729,
+      "memory(GiB)": 12.9,
+      "step": 775,
+      "train_speed(iter/s)": 0.079857
+    },
+    {
+      "acc": 0.81033506,
+      "epoch": 0.1001315831701916,
+      "grad_norm": 0.21427662670612335,
+      "learning_rate": 1.2658926150792322e-05,
+      "loss": 0.62097745,
+      "memory(GiB)": 12.9,
+      "step": 780,
+      "train_speed(iter/s)": 0.080004
+    },
+    {
+      "acc": 0.82130699,
+      "epoch": 0.10077345229307744,
+      "grad_norm": 0.20593635737895966,
+      "learning_rate": 1.2114256511983274e-05,
+      "loss": 0.60655851,
+      "memory(GiB)": 12.9,
+      "step": 785,
+      "train_speed(iter/s)": 0.080173
+    },
+    {
+      "acc": 0.82261295,
+      "epoch": 0.10141532141596328,
+      "grad_norm": 0.2052464336156845,
+      "learning_rate": 1.157994445715706e-05,
+      "loss": 0.61125717,
+      "memory(GiB)": 12.9,
+      "step": 790,
+      "train_speed(iter/s)": 0.080342
+    },
+    {
+      "acc": 0.82531328,
+      "epoch": 0.10205719053884912,
+      "grad_norm": 0.17600791156291962,
+      "learning_rate": 1.1056136061894384e-05,
+      "loss": 0.59510598,
+      "memory(GiB)": 12.9,
+      "step": 795,
+      "train_speed(iter/s)": 0.08049
+    },
+    {
+      "acc": 0.81779881,
+      "epoch": 0.10269905966173497,
+      "grad_norm": 0.2141779363155365,
+      "learning_rate": 1.0542974530180327e-05,
+      "loss": 0.63478656,
+      "memory(GiB)": 12.9,
+      "step": 800,
+      "train_speed(iter/s)": 0.080624
+    },
+    {
+      "epoch": 0.10269905966173497,
+      "eval_acc": 0.8207697400203976,
+      "eval_loss": 0.6077002882957458,
+      "eval_runtime": 194.8773,
+      "eval_samples_per_second": 6.455,
+      "eval_steps_per_second": 6.455,
+      "step": 800
+    },
+    {
+      "acc": 0.80902939,
+      "epoch": 0.10334092878462081,
+      "grad_norm": 0.2090810388326645,
+      "learning_rate": 1.0040600155253765e-05,
+      "loss": 0.66100531,
+      "memory(GiB)": 12.9,
+      "step": 805,
+      "train_speed(iter/s)": 0.079226
+    },
+    {
+      "acc": 0.81437035,
+      "epoch": 0.10398279790750665,
+      "grad_norm": 0.22697587311267853,
+      "learning_rate": 9.549150281252633e-06,
+      "loss": 0.63930345,
+      "memory(GiB)": 12.9,
+      "step": 810,
+      "train_speed(iter/s)": 0.079387
+    },
+    {
+      "acc": 0.81263914,
+      "epoch": 0.1046246670303925,
+      "grad_norm": 0.21495579183101654,
+      "learning_rate": 9.068759265665384e-06,
+      "loss": 0.63128233,
+      "memory(GiB)": 12.9,
+      "step": 815,
+      "train_speed(iter/s)": 0.079538
+    },
+    {
+      "acc": 0.80606184,
+      "epoch": 0.10526653615327834,
+      "grad_norm": 0.24026648700237274,
+      "learning_rate": 8.599558442598998e-06,
+      "loss": 0.66762576,
+      "memory(GiB)": 12.9,
+      "step": 820,
+      "train_speed(iter/s)": 0.07969
+    },
+    {
+      "acc": 0.83144102,
+      "epoch": 0.10590840527616419,
+      "grad_norm": 0.2179374396800995,
+      "learning_rate": 8.141676086873572e-06,
+      "loss": 0.56980219,
+      "memory(GiB)": 12.9,
+      "step": 825,
+      "train_speed(iter/s)": 0.079849
+    },
+    {
+      "acc": 0.81910725,
+      "epoch": 0.10655027439905003,
+      "grad_norm": 0.19496454298496246,
+      "learning_rate": 7.695237378953223e-06,
+      "loss": 0.61694736,
+      "memory(GiB)": 12.9,
+      "step": 830,
+      "train_speed(iter/s)": 0.07997
+    },
+    {
+      "acc": 0.81330662,
+      "epoch": 0.10719214352193587,
+      "grad_norm": 0.23187701404094696,
+      "learning_rate": 7.260364370723044e-06,
+      "loss": 0.64870877,
+      "memory(GiB)": 12.9,
+      "step": 835,
+      "train_speed(iter/s)": 0.080126
+    },
+    {
+      "acc": 0.81870842,
+      "epoch": 0.10783401264482172,
+      "grad_norm": 0.20400239527225494,
+      "learning_rate": 6.837175952121306e-06,
+      "loss": 0.61115222,
+      "memory(GiB)": 12.9,
+      "step": 840,
+      "train_speed(iter/s)": 0.080258
+    },
+    {
+      "acc": 0.82128296,
+      "epoch": 0.10847588176770756,
+      "grad_norm": 0.24608130753040314,
+      "learning_rate": 6.425787818636131e-06,
+      "loss": 0.61838484,
+      "memory(GiB)": 12.9,
+      "step": 845,
+      "train_speed(iter/s)": 0.080422
+    },
+    {
+      "acc": 0.8118104,
+      "epoch": 0.1091177508905934,
+      "grad_norm": 0.21192343533039093,
+      "learning_rate": 6.026312439675552e-06,
+      "loss": 0.67037339,
+      "memory(GiB)": 12.9,
+      "step": 850,
+      "train_speed(iter/s)": 0.080553
+    },
+    {
+      "epoch": 0.1091177508905934,
+      "eval_acc": 0.8208858520874116,
+      "eval_loss": 0.606462836265564,
+      "eval_runtime": 194.9575,
+      "eval_samples_per_second": 6.453,
+      "eval_steps_per_second": 6.453,
+      "step": 850
+    },
+    {
+      "acc": 0.81058187,
+      "epoch": 0.10975962001347925,
+      "grad_norm": 0.2256067544221878,
+      "learning_rate": 5.6388590278194096e-06,
+      "loss": 0.63853312,
+      "memory(GiB)": 12.9,
+      "step": 855,
+      "train_speed(iter/s)": 0.079236
+    },
+    {
+      "acc": 0.8174571,
+      "epoch": 0.11040148913636509,
+      "grad_norm": 0.21500875055789948,
+      "learning_rate": 5.263533508961827e-06,
+      "loss": 0.6172534,
+      "memory(GiB)": 12.9,
+      "step": 860,
+      "train_speed(iter/s)": 0.079369
+    },
+    {
+      "acc": 0.8093504,
+      "epoch": 0.11104335825925093,
+      "grad_norm": 0.23700734972953796,
+      "learning_rate": 4.900438493352055e-06,
+      "loss": 0.65724411,
+      "memory(GiB)": 12.9,
+      "step": 865,
+      "train_speed(iter/s)": 0.07952
+    },
+    {
+      "acc": 0.81735506,
+      "epoch": 0.11168522738213678,
+      "grad_norm": 0.21098878979682922,
+      "learning_rate": 4.549673247541875e-06,
+      "loss": 0.61837373,
+      "memory(GiB)": 12.9,
+      "step": 870,
+      "train_speed(iter/s)": 0.079659
+    },
+    {
+      "acc": 0.81503067,
+      "epoch": 0.11232709650502262,
+      "grad_norm": 0.20609669387340546,
+      "learning_rate": 4.2113336672471245e-06,
+      "loss": 0.61365747,
+      "memory(GiB)": 12.9,
+      "step": 875,
+      "train_speed(iter/s)": 0.079794
+    },
+    {
+      "acc": 0.80629053,
+      "epoch": 0.11296896562790847,
+      "grad_norm": 0.2180800437927246,
+      "learning_rate": 3.885512251130763e-06,
+      "loss": 0.66433964,
+      "memory(GiB)": 12.9,
+      "step": 880,
+      "train_speed(iter/s)": 0.079933
+    },
+    {
+      "acc": 0.82552185,
+      "epoch": 0.11361083475079431,
+      "grad_norm": 0.2074093371629715,
+      "learning_rate": 3.5722980755146517e-06,
+      "loss": 0.5967988,
+      "memory(GiB)": 12.9,
+      "step": 885,
+      "train_speed(iter/s)": 0.080078
+    },
+    {
+      "acc": 0.81032143,
+      "epoch": 0.11425270387368015,
+      "grad_norm": 0.232750803232193,
+      "learning_rate": 3.271776770026963e-06,
+      "loss": 0.66238441,
+      "memory(GiB)": 12.9,
+      "step": 890,
+      "train_speed(iter/s)": 0.080218
+    },
+    {
+      "acc": 0.8156251,
+      "epoch": 0.114894572996566,
+      "grad_norm": 0.24590888619422913,
+      "learning_rate": 2.9840304941919415e-06,
+      "loss": 0.6185041,
+      "memory(GiB)": 12.9,
+      "step": 895,
+      "train_speed(iter/s)": 0.080356
+    },
+    {
+      "acc": 0.81570625,
+      "epoch": 0.11553644211945184,
+      "grad_norm": 0.2058228850364685,
+      "learning_rate": 2.7091379149682685e-06,
+      "loss": 0.61333613,
+      "memory(GiB)": 12.9,
+      "step": 900,
+      "train_speed(iter/s)": 0.080474
+    },
+    {
+      "epoch": 0.11553644211945184,
+      "eval_acc": 0.8202886005724797,
+      "eval_loss": 0.6085067391395569,
+      "eval_runtime": 194.5686,
+      "eval_samples_per_second": 6.466,
+      "eval_steps_per_second": 6.466,
+      "step": 900
+    },
+    {
+      "acc": 0.81097269,
+      "epoch": 0.11617831124233768,
+      "grad_norm": 0.23131993412971497,
+      "learning_rate": 2.4471741852423237e-06,
+      "loss": 0.64109416,
+      "memory(GiB)": 12.9,
+      "step": 905,
+      "train_speed(iter/s)": 0.079218
+    },
+    {
+      "acc": 0.82361622,
+      "epoch": 0.11682018036522353,
+      "grad_norm": 0.18522906303405762,
+      "learning_rate": 2.1982109232821178e-06,
+      "loss": 0.59271107,
+      "memory(GiB)": 12.9,
+      "step": 910,
+      "train_speed(iter/s)": 0.079368
+    },
+    {
+      "acc": 0.81258793,
+      "epoch": 0.11746204948810937,
+      "grad_norm": 0.21756961941719055,
+      "learning_rate": 1.962316193157593e-06,
+      "loss": 0.65264983,
+      "memory(GiB)": 12.9,
+      "step": 915,
+      "train_speed(iter/s)": 0.07951
+    },
+    {
+      "acc": 0.82611704,
+      "epoch": 0.11810391861099521,
+      "grad_norm": 0.20278231799602509,
+      "learning_rate": 1.7395544861325718e-06,
+      "loss": 0.59875569,
+      "memory(GiB)": 12.9,
+      "step": 920,
+      "train_speed(iter/s)": 0.079655
+    },
+    {
+      "acc": 0.82435198,
+      "epoch": 0.11874578773388106,
+      "grad_norm": 0.21583449840545654,
+      "learning_rate": 1.5299867030334814e-06,
+      "loss": 0.62275109,
+      "memory(GiB)": 12.9,
+      "step": 925,
+      "train_speed(iter/s)": 0.079795
+    },
+    {
+      "acc": 0.81717892,
+      "epoch": 0.1193876568567669,
+      "grad_norm": 0.22039002180099487,
+      "learning_rate": 1.333670137599713e-06,
+      "loss": 0.61020494,
+      "memory(GiB)": 12.9,
+      "step": 930,
+      "train_speed(iter/s)": 0.079939
+    },
+    {
+      "acc": 0.80936031,
+      "epoch": 0.12002952597965275,
+      "grad_norm": 0.21585673093795776,
+      "learning_rate": 1.1506584608200367e-06,
+      "loss": 0.63481708,
+      "memory(GiB)": 12.9,
+      "step": 935,
+      "train_speed(iter/s)": 0.080073
+    },
+    {
+      "acc": 0.81828098,
+      "epoch": 0.12067139510253859,
+      "grad_norm": 0.22403886914253235,
+      "learning_rate": 9.810017062595322e-07,
+      "loss": 0.62871428,
+      "memory(GiB)": 12.9,
+      "step": 940,
+      "train_speed(iter/s)": 0.080211
+    },
+    {
+      "acc": 0.82708139,
+      "epoch": 0.12131326422542443,
+      "grad_norm": 0.22789925336837769,
+      "learning_rate": 8.247462563808817e-07,
+      "loss": 0.59829245,
+      "memory(GiB)": 12.9,
+      "step": 945,
+      "train_speed(iter/s)": 0.080336
+    },
+    {
+      "acc": 0.81116629,
+      "epoch": 0.12195513334831028,
+      "grad_norm": 0.22641904652118683,
+      "learning_rate": 6.819348298638839e-07,
+      "loss": 0.629322,
+      "memory(GiB)": 12.9,
+      "step": 950,
+      "train_speed(iter/s)": 0.080481
+    },
+    {
+      "epoch": 0.12195513334831028,
+      "eval_acc": 0.8201861908583086,
+      "eval_loss": 0.6082066297531128,
+      "eval_runtime": 194.2718,
+      "eval_samples_per_second": 6.475,
+      "eval_steps_per_second": 6.475,
+      "step": 950
+    },
+    {
+      "acc": 0.81035719,
+      "epoch": 0.12259700247119612,
+      "grad_norm": 0.21176832914352417,
+      "learning_rate": 5.526064699265753e-07,
+      "loss": 0.65304608,
+      "memory(GiB)": 12.9,
+      "step": 955,
+      "train_speed(iter/s)": 0.079321
+    },
+    {
+      "acc": 0.80980167,
+      "epoch": 0.12323887159408196,
+      "grad_norm": 0.23720386624336243,
+      "learning_rate": 4.367965336512403e-07,
+      "loss": 0.65838566,
+      "memory(GiB)": 12.9,
+      "step": 960,
+      "train_speed(iter/s)": 0.079457
+    },
+    {
+      "acc": 0.81608143,
+      "epoch": 0.12388074071696781,
+      "grad_norm": 0.19302093982696533,
+      "learning_rate": 3.3453668231809286e-07,
+      "loss": 0.64156389,
+      "memory(GiB)": 12.9,
+      "step": 965,
+      "train_speed(iter/s)": 0.079569
+    },
+    {
+      "acc": 0.81429224,
+      "epoch": 0.12452260983985365,
+      "grad_norm": 0.20239369571208954,
+      "learning_rate": 2.458548727494292e-07,
+      "loss": 0.63968878,
+      "memory(GiB)": 12.9,
+      "step": 970,
+      "train_speed(iter/s)": 0.079694
+    },
+    {
+      "acc": 0.80978498,
+      "epoch": 0.1251644789627395,
+      "grad_norm": 0.20085211098194122,
+      "learning_rate": 1.7077534966650766e-07,
+      "loss": 0.65152073,
+      "memory(GiB)": 12.9,
+      "step": 975,
+      "train_speed(iter/s)": 0.079826
+    },
+    {
+      "acc": 0.82283268,
+      "epoch": 0.12580634808562535,
+      "grad_norm": 0.23421017825603485,
+      "learning_rate": 1.0931863906127327e-07,
+      "loss": 0.59241028,
+      "memory(GiB)": 12.9,
+      "step": 980,
+      "train_speed(iter/s)": 0.079952
+    },
+    {
+      "acc": 0.81115065,
+      "epoch": 0.12644821720851118,
+      "grad_norm": 0.19245800375938416,
+      "learning_rate": 6.150154258476315e-08,
+      "loss": 0.64031863,
+      "memory(GiB)": 12.9,
+      "step": 985,
+      "train_speed(iter/s)": 0.080066
+    },
+    {
+      "acc": 0.81750393,
+      "epoch": 0.12709008633139704,
+      "grad_norm": 0.21410977840423584,
+      "learning_rate": 2.7337132953697554e-08,
+      "loss": 0.63289447,
+      "memory(GiB)": 12.9,
+      "step": 990,
+      "train_speed(iter/s)": 0.080194
+    },
+    {
+      "acc": 0.80839672,
+      "epoch": 0.12773195545428287,
+      "grad_norm": 0.24657242000102997,
+      "learning_rate": 6.834750376549792e-09,
+      "loss": 0.67404447,
+      "memory(GiB)": 12.9,
+      "step": 995,
+      "train_speed(iter/s)": 0.080318
+    },
+    {
+      "acc": 0.82744455,
+      "epoch": 0.12837382457716873,
+      "grad_norm": 0.20574873685836792,
+      "learning_rate": 0.0,
+      "loss": 0.58532543,
+      "memory(GiB)": 12.9,
+      "step": 1000,
+      "train_speed(iter/s)": 0.080433
+    },
+    {
+      "epoch": 0.12837382457716873,
+      "eval_acc": 0.8205681371523807,
+      "eval_loss": 0.6072012186050415,
+      "eval_runtime": 194.0316,
+      "eval_samples_per_second": 6.483,
+      "eval_steps_per_second": 6.483,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.855587826431877e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5ca841f7bd4a855ee575421abc61a664c4a5bac12d5a654d3adfe5e4f1cc45d
+size 7480

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed32b9bb4a7994322f82959b3c889e3d5fcf6df641a4cf87d551f2291096bd65
+size 50792328

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/additional_config.json

@@ -0,0 +1 @@
+{"lora_dtype": null, "lorap_lr_ratio": null, "lorap_emb_lr": 1e-06}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/configuration.json

@@ -0,0 +1,11 @@
+{
+    "adapter_cfg": {
+        "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+        "model_revision": "main",
+        "sft_type": "lora",
+        "tuner_backend": "peft",
+        "template_type": "phi3-vl",
+        "dtype": "bf16",
+        "system": null
+    }
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 32000,
+  "max_new_tokens": 2048,
+  "pad_token_id": 32000,
+  "transformers_version": "4.46.3"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d5b36d15d3b4cb5d6ac6c45668dd7a4e234996cf5b5bcfa427d6dcfc9581fe0
+size 100878458

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f993860293e99c2d3e78d865fcaaf179473939fc9348266d1308c749068004b
+size 14244

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:933907a28d624df5625600b77cfaf69aac83ec99919cf400ec983afa16156fea
+size 1064

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/sft_args.json

@@ -0,0 +1,246 @@
+{
+  "model_type": "phi3_5-vision-instruct",
+  "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "model_revision": "main",
+  "full_determinism": false,
+  "sft_type": "lora",
+  "freeze_parameters": [],
+  "freeze_vit": false,
+  "freeze_parameters_ratio": 0.0,
+  "additional_trainable_parameters": [],
+  "tuner_backend": "peft",
+  "template_type": "phi3-vl",
+  "output_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713",
+  "add_output_dir_suffix": true,
+  "ddp_backend": null,
+  "ddp_find_unused_parameters": null,
+  "ddp_broadcast_buffers": null,
+  "ddp_timeout": 1800,
+  "seed": 42,
+  "resume_from_checkpoint": null,
+  "resume_only_model": false,
+  "ignore_data_skip": false,
+  "dtype": "bf16",
+  "packing": false,
+  "train_backend": "transformers",
+  "tp": 1,
+  "pp": 1,
+  "min_lr": null,
+  "sequence_parallel": false,
+  "model_kwargs": {},
+  "loss_name": null,
+  "dataset": [
+    "llamafactory/OpenO1-SFT"
+  ],
+  "val_dataset": [],
+  "dataset_seed": 42,
+  "dataset_test_ratio": 0.01,
+  "use_loss_scale": false,
+  "loss_scale_config_path": "/home/pzcmoe/home/zhc/llama/swift/swift/llm/agent/default_loss_scale_config.json",
+  "system": null,
+  "tools_prompt": "react_en",
+  "max_length": 2048,
+  "truncation_strategy": "delete",
+  "check_dataset_strategy": "none",
+  "streaming": false,
+  "streaming_val_size": 0,
+  "streaming_buffer_size": 16384,
+  "model_name": [
+    null,
+    null
+  ],
+  "model_author": [
+    null,
+    null
+  ],
+  "quant_method": null,
+  "quantization_bit": 0,
+  "hqq_axis": 0,
+  "hqq_dynamic_config_path": null,
+  "bnb_4bit_comp_dtype": "bf16",
+  "bnb_4bit_quant_type": "nf4",
+  "bnb_4bit_use_double_quant": true,
+  "bnb_4bit_quant_storage": null,
+  "rescale_image": -1,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "target_regex": null,
+  "modules_to_save": [],
+  "lora_rank": 8,
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "lora_bias_trainable": "none",
+  "lora_dtype": null,
+  "lora_lr_ratio": null,
+  "use_rslora": false,
+  "use_dora": false,
+  "init_lora_weights": true,
+  "fourier_n_frequency": 2000,
+  "fourier_scaling": 300.0,
+  "rope_scaling": null,
+  "boft_block_size": 4,
+  "boft_block_num": 0,
+  "boft_n_butterfly_factor": 1,
+  "boft_dropout": 0.0,
+  "vera_rank": 256,
+  "vera_projection_prng_key": 0,
+  "vera_dropout": 0.0,
+  "vera_d_initial": 0.1,
+  "adapter_act": "gelu",
+  "adapter_length": 128,
+  "use_galore": false,
+  "galore_target_modules": null,
+  "galore_rank": 128,
+  "galore_update_proj_gap": 50,
+  "galore_scale": 1.0,
+  "galore_proj_type": "std",
+  "galore_optim_per_parameter": false,
+  "galore_with_embedding": false,
+  "galore_quantization": false,
+  "galore_proj_quant": false,
+  "galore_proj_bits": 4,
+  "galore_proj_group_size": 256,
+  "galore_cos_threshold": 0.4,
+  "galore_gamma_proj": 2,
+  "galore_queue_size": 5,
+  "adalora_target_r": 8,
+  "adalora_init_r": 12,
+  "adalora_tinit": 0,
+  "adalora_tfinal": 0,
+  "adalora_deltaT": 1,
+  "adalora_beta1": 0.85,
+  "adalora_beta2": 0.85,
+  "adalora_orth_reg_weight": 0.5,
+  "ia3_feedforward_modules": [],
+  "llamapro_num_new_blocks": 4,
+  "llamapro_num_groups": null,
+  "neftune_noise_alpha": null,
+  "neftune_backend": "transformers",
+  "lisa_activated_layers": 0,
+  "lisa_step_interval": 20,
+  "reft_layer_key": null,
+  "reft_layers": null,
+  "reft_rank": 4,
+  "reft_intervention_type": "LoreftIntervention",
+  "reft_args": null,
+  "use_liger": false,
+  "gradient_checkpointing": true,
+  "vit_use_gc": true,
+  "deepspeed": null,
+  "batch_size": 1,
+  "eval_batch_size": 1,
+  "auto_find_batch_size": false,
+  "num_train_epochs": 1,
+  "max_steps": 1000,
+  "optim": "adamw_torch",
+  "adam_beta1": 0.9,
+  "adam_beta2": 0.95,
+  "adam_epsilon": 1e-08,
+  "learning_rate": 0.0001,
+  "weight_decay": 0.1,
+  "gradient_accumulation_steps": 16,
+  "max_grad_norm": 1,
+  "predict_with_generate": false,
+  "lr_scheduler_type": "cosine",
+  "lr_scheduler_kwargs": {},
+  "warmup_ratio": 0.05,
+  "warmup_steps": 0,
+  "eval_steps": 50,
+  "save_steps": 50,
+  "save_only_model": false,
+  "save_total_limit": null,
+  "logging_steps": 5,
+  "acc_steps": 1,
+  "dataloader_num_workers": 1,
+  "dataloader_pin_memory": true,
+  "dataloader_drop_last": false,
+  "push_to_hub": false,
+  "hub_model_id": null,
+  "hub_token": null,
+  "hub_private_repo": false,
+  "hub_strategy": "every_save",
+  "test_oom_error": false,
+  "disable_tqdm": false,
+  "lazy_tokenize": true,
+  "preprocess_num_proc": 1,
+  "use_flash_attn": null,
+  "ignore_args_error": false,
+  "check_model_is_latest": true,
+  "logging_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs",
+  "report_to": [
+    "tensorboard"
+  ],
+  "acc_strategy": "token",
+  "save_on_each_node": false,
+  "evaluation_strategy": "steps",
+  "save_strategy": "steps",
+  "save_safetensors": true,
+  "gpu_memory_fraction": null,
+  "include_num_input_tokens_seen": false,
+  "local_repo_path": null,
+  "custom_register_path": null,
+  "custom_dataset_info": null,
+  "device_map_config": null,
+  "device_max_memory": [],
+  "max_new_tokens": 2048,
+  "do_sample": null,
+  "temperature": null,
+  "top_k": null,
+  "top_p": null,
+  "repetition_penalty": null,
+  "num_beams": 1,
+  "fsdp": "",
+  "fsdp_config": null,
+  "sequence_parallel_size": 1,
+  "model_layer_cls_name": null,
+  "metric_warmup_step": 0,
+  "fsdp_num": 1,
+  "per_device_train_batch_size": null,
+  "per_device_eval_batch_size": null,
+  "eval_strategy": null,
+  "self_cognition_sample": 0,
+  "train_dataset_mix_ratio": 0.0,
+  "train_dataset_mix_ds": [
+    "ms-bench"
+  ],
+  "train_dataset_sample": -1,
+  "val_dataset_sample": null,
+  "safe_serialization": null,
+  "only_save_model": null,
+  "neftune_alpha": null,
+  "deepspeed_config_path": null,
+  "model_cache_dir": null,
+  "lora_dropout_p": null,
+  "lora_target_modules": [],
+  "lora_target_regex": null,
+  "lora_modules_to_save": [],
+  "boft_target_modules": [],
+  "boft_modules_to_save": [],
+  "vera_target_modules": [],
+  "vera_modules_to_save": [],
+  "ia3_target_modules": [],
+  "ia3_modules_to_save": [],
+  "custom_train_dataset_path": [],
+  "custom_val_dataset_path": [],
+  "device_map_config_path": null,
+  "push_hub_strategy": null,
+  "use_self_cognition": false,
+  "is_multimodal": true,
+  "is_vision": true,
+  "lora_use_embedding": false,
+  "lora_use_all": false,
+  "lora_m2s_use_embedding": false,
+  "lora_m2s_use_ln": false,
+  "torch_dtype": "torch.bfloat16",
+  "fp16": false,
+  "bf16": true,
+  "rank": -1,
+  "local_rank": -1,
+  "world_size": 1,
+  "local_world_size": 1,
+  "bnb_4bit_compute_dtype": "torch.bfloat16",
+  "load_in_4bit": false,
+  "load_in_8bit": false,
+  "train_sampler_random": true,
+  "train_type": "sft",
+  "training_args": "Seq2SeqTrainingArguments(output_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, eval_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=1, per_device_eval_batch_size=1, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=16, eval_accumulation_steps=None, eval_delay=0, torch_empty_cache_steps=None, learning_rate=0.0001, weight_decay=0.1, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, max_grad_norm=1, num_train_epochs=1, max_steps=1000, lr_scheduler_type=<SchedulerType.COSINE: 'cosine'>, lr_scheduler_kwargs={}, warmup_ratio=0.05, warmup_steps=0, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=True, logging_steps=5, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=50, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=False, restore_callback_states_from_checkpoint=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=42, jit_mode_eval=False, use_ipex=False, bf16=True, fp16=False, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=50, dataloader_num_workers=1, dataloader_prefetch_factor=None, past_index=-1, run_name='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', disable_tqdm=False, remove_unused_columns=False, label_names=None, load_best_model_at_end=False, metric_for_best_model='loss', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, accelerator_config=AcceleratorConfig(split_batches=False, dispatch_batches=False, even_batches=True, use_seedable_sampler=True, non_blocking=False, gradient_accumulation_kwargs=None, use_configured_state=False), deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=False, resume_from_checkpoint=None, hub_model_id=None, hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, include_for_metrics=[], eval_do_concat_batches=True, fp16_backend='auto', evaluation_strategy=None, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=None, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=None, optim_target_modules=None, batch_eval_metrics=False, eval_on_start=False, use_liger_kernel=False, eval_use_gather_object=False, average_tokens_across_devices=False, sortish_sampler=False, predict_with_generate=False, generation_max_length=None, generation_num_beams=None, generation_config=GenerationConfig {\n  \"bos_token_id\": 1,\n  \"eos_token_id\": 32000,\n  \"max_new_tokens\": 2048,\n  \"pad_token_id\": 32000\n}\n, acc_strategy='token', loss_name=None, additional_saved_files=[], train_sampler_random=True, metric_warmup_step=0, train_dataset_sample=-1)"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/trainer_state.json

@@ -0,0 +1,370 @@
+{
+  "best_metric": 0.63503182,
+  "best_model_checkpoint": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150",
+  "epoch": 0.019256073686575306,
+  "eval_steps": 50,
+  "global_step": 150,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "acc": 0.77952504,
+      "epoch": 0.0001283738245771687,
+      "grad_norm": 0.15511861443519592,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 0.82634771,
+      "memory(GiB)": 11.42,
+      "step": 1,
+      "train_speed(iter/s)": 0.092016
+    },
+    {
+      "acc": 0.77674872,
+      "epoch": 0.0006418691228858435,
+      "grad_norm": 0.14591480791568756,
+      "learning_rate": 1e-05,
+      "loss": 0.8159712,
+      "memory(GiB)": 12.15,
+      "step": 5,
+      "train_speed(iter/s)": 0.10552
+    },
+    {
+      "acc": 0.77379012,
+      "epoch": 0.001283738245771687,
+      "grad_norm": 0.151850163936615,
+      "learning_rate": 2e-05,
+      "loss": 0.8015564,
+      "memory(GiB)": 12.15,
+      "step": 10,
+      "train_speed(iter/s)": 0.107013
+    },
+    {
+      "acc": 0.7784133,
+      "epoch": 0.0019256073686575307,
+      "grad_norm": 0.15997682511806488,
+      "learning_rate": 3e-05,
+      "loss": 0.79284344,
+      "memory(GiB)": 12.15,
+      "step": 15,
+      "train_speed(iter/s)": 0.110995
+    },
+    {
+      "acc": 0.7815052,
+      "epoch": 0.002567476491543374,
+      "grad_norm": 0.14427673816680908,
+      "learning_rate": 4e-05,
+      "loss": 0.78122258,
+      "memory(GiB)": 12.9,
+      "step": 20,
+      "train_speed(iter/s)": 0.111462
+    },
+    {
+      "acc": 0.7915082,
+      "epoch": 0.0032093456144292177,
+      "grad_norm": 0.1590222716331482,
+      "learning_rate": 5e-05,
+      "loss": 0.7404335,
+      "memory(GiB)": 12.9,
+      "step": 25,
+      "train_speed(iter/s)": 0.112268
+    },
+    {
+      "acc": 0.80291214,
+      "epoch": 0.0038512147373150614,
+      "grad_norm": 0.13638992607593536,
+      "learning_rate": 6e-05,
+      "loss": 0.68063788,
+      "memory(GiB)": 12.9,
+      "step": 30,
+      "train_speed(iter/s)": 0.111517
+    },
+    {
+      "acc": 0.79923964,
+      "epoch": 0.0044930838602009054,
+      "grad_norm": 0.1271245777606964,
+      "learning_rate": 7e-05,
+      "loss": 0.68800621,
+      "memory(GiB)": 12.9,
+      "step": 35,
+      "train_speed(iter/s)": 0.110574
+    },
+    {
+      "acc": 0.80240812,
+      "epoch": 0.005134952983086748,
+      "grad_norm": 0.11937133222818375,
+      "learning_rate": 8e-05,
+      "loss": 0.69939818,
+      "memory(GiB)": 12.9,
+      "step": 40,
+      "train_speed(iter/s)": 0.109469
+    },
+    {
+      "acc": 0.78317132,
+      "epoch": 0.005776822105972592,
+      "grad_norm": 0.1665358543395996,
+      "learning_rate": 9e-05,
+      "loss": 0.75735736,
+      "memory(GiB)": 12.9,
+      "step": 45,
+      "train_speed(iter/s)": 0.108968
+    },
+    {
+      "acc": 0.80379133,
+      "epoch": 0.0064186912288584355,
+      "grad_norm": 0.14883440732955933,
+      "learning_rate": 0.0001,
+      "loss": 0.6824338,
+      "memory(GiB)": 12.9,
+      "step": 50,
+      "train_speed(iter/s)": 0.109163
+    },
+    {
+      "epoch": 0.0064186912288584355,
+      "eval_acc": 0.8081346555994846,
+      "eval_loss": 0.6729312539100647,
+      "eval_runtime": 195.4613,
+      "eval_samples_per_second": 6.436,
+      "eval_steps_per_second": 6.436,
+      "step": 50
+    },
+    {
+      "acc": 0.7984889,
+      "epoch": 0.007060560351744279,
+      "grad_norm": 0.16687534749507904,
+      "learning_rate": 9.999316524962345e-05,
+      "loss": 0.7141089,
+      "memory(GiB)": 12.9,
+      "step": 55,
+      "train_speed(iter/s)": 0.078646
+    },
+    {
+      "acc": 0.79216175,
+      "epoch": 0.007702429474630123,
+      "grad_norm": 0.17546051740646362,
+      "learning_rate": 9.997266286704631e-05,
+      "loss": 0.72419305,
+      "memory(GiB)": 12.9,
+      "step": 60,
+      "train_speed(iter/s)": 0.080466
+    },
+    {
+      "acc": 0.80358973,
+      "epoch": 0.008344298597515967,
+      "grad_norm": 0.16079305112361908,
+      "learning_rate": 9.993849845741524e-05,
+      "loss": 0.67717872,
+      "memory(GiB)": 12.9,
+      "step": 65,
+      "train_speed(iter/s)": 0.082252
+    },
+    {
+      "acc": 0.80615339,
+      "epoch": 0.008986167720401811,
+      "grad_norm": 0.1396234929561615,
+      "learning_rate": 9.989068136093873e-05,
+      "loss": 0.70638905,
+      "memory(GiB)": 12.9,
+      "step": 70,
+      "train_speed(iter/s)": 0.084
+    },
+    {
+      "acc": 0.79676814,
+      "epoch": 0.009628036843287653,
+      "grad_norm": 0.1340462565422058,
+      "learning_rate": 9.98292246503335e-05,
+      "loss": 0.71420946,
+      "memory(GiB)": 12.9,
+      "step": 75,
+      "train_speed(iter/s)": 0.08542
+    },
+    {
+      "acc": 0.79325166,
+      "epoch": 0.010269905966173496,
+      "grad_norm": 0.13259001076221466,
+      "learning_rate": 9.975414512725057e-05,
+      "loss": 0.71777782,
+      "memory(GiB)": 12.9,
+      "step": 80,
+      "train_speed(iter/s)": 0.086655
+    },
+    {
+      "acc": 0.81705523,
+      "epoch": 0.01091177508905934,
+      "grad_norm": 0.15833652019500732,
+      "learning_rate": 9.966546331768191e-05,
+      "loss": 0.62526927,
+      "memory(GiB)": 12.9,
+      "step": 85,
+      "train_speed(iter/s)": 0.088014
+    },
+    {
+      "acc": 0.80851898,
+      "epoch": 0.011553644211945184,
+      "grad_norm": 0.1456330418586731,
+      "learning_rate": 9.956320346634876e-05,
+      "loss": 0.68010745,
+      "memory(GiB)": 12.9,
+      "step": 90,
+      "train_speed(iter/s)": 0.088941
+    },
+    {
+      "acc": 0.81903477,
+      "epoch": 0.012195513334831027,
+      "grad_norm": 0.14285945892333984,
+      "learning_rate": 9.944739353007344e-05,
+      "loss": 0.6090106,
+      "memory(GiB)": 12.9,
+      "step": 95,
+      "train_speed(iter/s)": 0.089902
+    },
+    {
+      "acc": 0.82483721,
+      "epoch": 0.012837382457716871,
+      "grad_norm": 0.1478744000196457,
+      "learning_rate": 9.931806517013612e-05,
+      "loss": 0.59807158,
+      "memory(GiB)": 12.9,
+      "step": 100,
+      "train_speed(iter/s)": 0.090663
+    },
+    {
+      "epoch": 0.012837382457716871,
+      "eval_acc": 0.813768508772143,
+      "eval_loss": 0.645352840423584,
+      "eval_runtime": 195.755,
+      "eval_samples_per_second": 6.426,
+      "eval_steps_per_second": 6.426,
+      "step": 100
+    },
+    {
+      "acc": 0.80671978,
+      "epoch": 0.013479251580602715,
+      "grad_norm": 0.15444877743721008,
+      "learning_rate": 9.917525374361912e-05,
+      "loss": 0.66537051,
+      "memory(GiB)": 12.9,
+      "step": 105,
+      "train_speed(iter/s)": 0.078062
+    },
+    {
+      "acc": 0.81962786,
+      "epoch": 0.014121120703488558,
+      "grad_norm": 0.16826578974723816,
+      "learning_rate": 9.901899829374047e-05,
+      "loss": 0.60835891,
+      "memory(GiB)": 12.9,
+      "step": 110,
+      "train_speed(iter/s)": 0.079089
+    },
+    {
+      "acc": 0.80644588,
+      "epoch": 0.014762989826374402,
+      "grad_norm": 0.1519591063261032,
+      "learning_rate": 9.884934153917997e-05,
+      "loss": 0.66763787,
+      "memory(GiB)": 12.9,
+      "step": 115,
+      "train_speed(iter/s)": 0.08012
+    },
+    {
+      "acc": 0.8235981,
+      "epoch": 0.015404858949260246,
+      "grad_norm": 0.20040030777454376,
+      "learning_rate": 9.86663298624003e-05,
+      "loss": 0.5919826,
+      "memory(GiB)": 12.9,
+      "step": 120,
+      "train_speed(iter/s)": 0.081125
+    },
+    {
+      "acc": 0.81199884,
+      "epoch": 0.01604672807214609,
+      "grad_norm": 0.1842738389968872,
+      "learning_rate": 9.847001329696653e-05,
+      "loss": 0.63266978,
+      "memory(GiB)": 12.9,
+      "step": 125,
+      "train_speed(iter/s)": 0.082093
+    },
+    {
+      "acc": 0.81563005,
+      "epoch": 0.016688597195031935,
+      "grad_norm": 0.15375380218029022,
+      "learning_rate": 9.826044551386744e-05,
+      "loss": 0.61709371,
+      "memory(GiB)": 12.9,
+      "step": 130,
+      "train_speed(iter/s)": 0.082937
+    },
+    {
+      "acc": 0.80918331,
+      "epoch": 0.017330466317917778,
+      "grad_norm": 0.14841115474700928,
+      "learning_rate": 9.803768380684242e-05,
+      "loss": 0.63595695,
+      "memory(GiB)": 12.9,
+      "step": 135,
+      "train_speed(iter/s)": 0.083738
+    },
+    {
+      "acc": 0.81229877,
+      "epoch": 0.017972335440803622,
+      "grad_norm": 0.16627077758312225,
+      "learning_rate": 9.780178907671789e-05,
+      "loss": 0.64757662,
+      "memory(GiB)": 12.9,
+      "step": 140,
+      "train_speed(iter/s)": 0.084642
+    },
+    {
+      "acc": 0.804142,
+      "epoch": 0.018614204563689465,
+      "grad_norm": 0.1714244931936264,
+      "learning_rate": 9.755282581475769e-05,
+      "loss": 0.6832706,
+      "memory(GiB)": 12.9,
+      "step": 145,
+      "train_speed(iter/s)": 0.085406
+    },
+    {
+      "acc": 0.79688311,
+      "epoch": 0.019256073686575306,
+      "grad_norm": 0.18916593492031097,
+      "learning_rate": 9.729086208503174e-05,
+      "loss": 0.70668921,
+      "memory(GiB)": 12.9,
+      "step": 150,
+      "train_speed(iter/s)": 0.086183
+    },
+    {
+      "epoch": 0.019256073686575306,
+      "eval_acc": 0.8153510047395746,
+      "eval_loss": 0.6350318193435669,
+      "eval_runtime": 196.0265,
+      "eval_samples_per_second": 6.417,
+      "eval_steps_per_second": 6.417,
+      "step": 150
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.844637904195174e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-150/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5ca841f7bd4a855ee575421abc61a664c4a5bac12d5a654d3adfe5e4f1cc45d
+size 7480

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d844e37c92e6154fc82860952fa7e91a623164af249cd9e431e3f7e2028e23d4
+size 50792328

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/additional_config.json

@@ -0,0 +1 @@
+{"lora_dtype": null, "lorap_lr_ratio": null, "lorap_emb_lr": 1e-06}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/configuration.json

@@ -0,0 +1,11 @@
+{
+    "adapter_cfg": {
+        "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+        "model_revision": "main",
+        "sft_type": "lora",
+        "tuner_backend": "peft",
+        "template_type": "phi3-vl",
+        "dtype": "bf16",
+        "system": null
+    }
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 32000,
+  "max_new_tokens": 2048,
+  "pad_token_id": 32000,
+  "transformers_version": "4.46.3"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1711fa8982f5b6fa02df8213057cc9e18d11d7fa2d89a4fc9aa20d86d7790023
+size 100878458

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab89609811018c79003d804395390bda2de480fa77651cf4c7d56ad6035a26f0
+size 14244

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:990a1cf86aba2c4ef745bb4fd8a9b7f25fcd3e01d84b539560e7d0b4ac6b96da
+size 1064

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/sft_args.json

@@ -0,0 +1,246 @@
+{
+  "model_type": "phi3_5-vision-instruct",
+  "model_id_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "model_revision": "main",
+  "full_determinism": false,
+  "sft_type": "lora",
+  "freeze_parameters": [],
+  "freeze_vit": false,
+  "freeze_parameters_ratio": 0.0,
+  "additional_trainable_parameters": [],
+  "tuner_backend": "peft",
+  "template_type": "phi3-vl",
+  "output_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713",
+  "add_output_dir_suffix": true,
+  "ddp_backend": null,
+  "ddp_find_unused_parameters": null,
+  "ddp_broadcast_buffers": null,
+  "ddp_timeout": 1800,
+  "seed": 42,
+  "resume_from_checkpoint": null,
+  "resume_only_model": false,
+  "ignore_data_skip": false,
+  "dtype": "bf16",
+  "packing": false,
+  "train_backend": "transformers",
+  "tp": 1,
+  "pp": 1,
+  "min_lr": null,
+  "sequence_parallel": false,
+  "model_kwargs": {},
+  "loss_name": null,
+  "dataset": [
+    "llamafactory/OpenO1-SFT"
+  ],
+  "val_dataset": [],
+  "dataset_seed": 42,
+  "dataset_test_ratio": 0.01,
+  "use_loss_scale": false,
+  "loss_scale_config_path": "/home/pzcmoe/home/zhc/llama/swift/swift/llm/agent/default_loss_scale_config.json",
+  "system": null,
+  "tools_prompt": "react_en",
+  "max_length": 2048,
+  "truncation_strategy": "delete",
+  "check_dataset_strategy": "none",
+  "streaming": false,
+  "streaming_val_size": 0,
+  "streaming_buffer_size": 16384,
+  "model_name": [
+    null,
+    null
+  ],
+  "model_author": [
+    null,
+    null
+  ],
+  "quant_method": null,
+  "quantization_bit": 0,
+  "hqq_axis": 0,
+  "hqq_dynamic_config_path": null,
+  "bnb_4bit_comp_dtype": "bf16",
+  "bnb_4bit_quant_type": "nf4",
+  "bnb_4bit_use_double_quant": true,
+  "bnb_4bit_quant_storage": null,
+  "rescale_image": -1,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "target_regex": null,
+  "modules_to_save": [],
+  "lora_rank": 8,
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "lora_bias_trainable": "none",
+  "lora_dtype": null,
+  "lora_lr_ratio": null,
+  "use_rslora": false,
+  "use_dora": false,
+  "init_lora_weights": true,
+  "fourier_n_frequency": 2000,
+  "fourier_scaling": 300.0,
+  "rope_scaling": null,
+  "boft_block_size": 4,
+  "boft_block_num": 0,
+  "boft_n_butterfly_factor": 1,
+  "boft_dropout": 0.0,
+  "vera_rank": 256,
+  "vera_projection_prng_key": 0,
+  "vera_dropout": 0.0,
+  "vera_d_initial": 0.1,
+  "adapter_act": "gelu",
+  "adapter_length": 128,
+  "use_galore": false,
+  "galore_target_modules": null,
+  "galore_rank": 128,
+  "galore_update_proj_gap": 50,
+  "galore_scale": 1.0,
+  "galore_proj_type": "std",
+  "galore_optim_per_parameter": false,
+  "galore_with_embedding": false,
+  "galore_quantization": false,
+  "galore_proj_quant": false,
+  "galore_proj_bits": 4,
+  "galore_proj_group_size": 256,
+  "galore_cos_threshold": 0.4,
+  "galore_gamma_proj": 2,
+  "galore_queue_size": 5,
+  "adalora_target_r": 8,
+  "adalora_init_r": 12,
+  "adalora_tinit": 0,
+  "adalora_tfinal": 0,
+  "adalora_deltaT": 1,
+  "adalora_beta1": 0.85,
+  "adalora_beta2": 0.85,
+  "adalora_orth_reg_weight": 0.5,
+  "ia3_feedforward_modules": [],
+  "llamapro_num_new_blocks": 4,
+  "llamapro_num_groups": null,
+  "neftune_noise_alpha": null,
+  "neftune_backend": "transformers",
+  "lisa_activated_layers": 0,
+  "lisa_step_interval": 20,
+  "reft_layer_key": null,
+  "reft_layers": null,
+  "reft_rank": 4,
+  "reft_intervention_type": "LoreftIntervention",
+  "reft_args": null,
+  "use_liger": false,
+  "gradient_checkpointing": true,
+  "vit_use_gc": true,
+  "deepspeed": null,
+  "batch_size": 1,
+  "eval_batch_size": 1,
+  "auto_find_batch_size": false,
+  "num_train_epochs": 1,
+  "max_steps": 1000,
+  "optim": "adamw_torch",
+  "adam_beta1": 0.9,
+  "adam_beta2": 0.95,
+  "adam_epsilon": 1e-08,
+  "learning_rate": 0.0001,
+  "weight_decay": 0.1,
+  "gradient_accumulation_steps": 16,
+  "max_grad_norm": 1,
+  "predict_with_generate": false,
+  "lr_scheduler_type": "cosine",
+  "lr_scheduler_kwargs": {},
+  "warmup_ratio": 0.05,
+  "warmup_steps": 0,
+  "eval_steps": 50,
+  "save_steps": 50,
+  "save_only_model": false,
+  "save_total_limit": null,
+  "logging_steps": 5,
+  "acc_steps": 1,
+  "dataloader_num_workers": 1,
+  "dataloader_pin_memory": true,
+  "dataloader_drop_last": false,
+  "push_to_hub": false,
+  "hub_model_id": null,
+  "hub_token": null,
+  "hub_private_repo": false,
+  "hub_strategy": "every_save",
+  "test_oom_error": false,
+  "disable_tqdm": false,
+  "lazy_tokenize": true,
+  "preprocess_num_proc": 1,
+  "use_flash_attn": null,
+  "ignore_args_error": false,
+  "check_model_is_latest": true,
+  "logging_dir": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs",
+  "report_to": [
+    "tensorboard"
+  ],
+  "acc_strategy": "token",
+  "save_on_each_node": false,
+  "evaluation_strategy": "steps",
+  "save_strategy": "steps",
+  "save_safetensors": true,
+  "gpu_memory_fraction": null,
+  "include_num_input_tokens_seen": false,
+  "local_repo_path": null,
+  "custom_register_path": null,
+  "custom_dataset_info": null,
+  "device_map_config": null,
+  "device_max_memory": [],
+  "max_new_tokens": 2048,
+  "do_sample": null,
+  "temperature": null,
+  "top_k": null,
+  "top_p": null,
+  "repetition_penalty": null,
+  "num_beams": 1,
+  "fsdp": "",
+  "fsdp_config": null,
+  "sequence_parallel_size": 1,
+  "model_layer_cls_name": null,
+  "metric_warmup_step": 0,
+  "fsdp_num": 1,
+  "per_device_train_batch_size": null,
+  "per_device_eval_batch_size": null,
+  "eval_strategy": null,
+  "self_cognition_sample": 0,
+  "train_dataset_mix_ratio": 0.0,
+  "train_dataset_mix_ds": [
+    "ms-bench"
+  ],
+  "train_dataset_sample": -1,
+  "val_dataset_sample": null,
+  "safe_serialization": null,
+  "only_save_model": null,
+  "neftune_alpha": null,
+  "deepspeed_config_path": null,
+  "model_cache_dir": null,
+  "lora_dropout_p": null,
+  "lora_target_modules": [],
+  "lora_target_regex": null,
+  "lora_modules_to_save": [],
+  "boft_target_modules": [],
+  "boft_modules_to_save": [],
+  "vera_target_modules": [],
+  "vera_modules_to_save": [],
+  "ia3_target_modules": [],
+  "ia3_modules_to_save": [],
+  "custom_train_dataset_path": [],
+  "custom_val_dataset_path": [],
+  "device_map_config_path": null,
+  "push_hub_strategy": null,
+  "use_self_cognition": false,
+  "is_multimodal": true,
+  "is_vision": true,
+  "lora_use_embedding": false,
+  "lora_use_all": false,
+  "lora_m2s_use_embedding": false,
+  "lora_m2s_use_ln": false,
+  "torch_dtype": "torch.bfloat16",
+  "fp16": false,
+  "bf16": true,
+  "rank": -1,
+  "local_rank": -1,
+  "world_size": 1,
+  "local_world_size": 1,
+  "bnb_4bit_compute_dtype": "torch.bfloat16",
+  "load_in_4bit": false,
+  "load_in_8bit": false,
+  "train_sampler_random": true,
+  "train_type": "sft",
+  "training_args": "Seq2SeqTrainingArguments(output_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, eval_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=1, per_device_eval_batch_size=1, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=16, eval_accumulation_steps=None, eval_delay=0, torch_empty_cache_steps=None, learning_rate=0.0001, weight_decay=0.1, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, max_grad_norm=1, num_train_epochs=1, max_steps=1000, lr_scheduler_type=<SchedulerType.COSINE: 'cosine'>, lr_scheduler_kwargs={}, warmup_ratio=0.05, warmup_steps=0, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/runs', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=True, logging_steps=5, logging_nan_inf_filter=True, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=50, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=False, restore_callback_states_from_checkpoint=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=42, jit_mode_eval=False, use_ipex=False, bf16=True, fp16=False, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=50, dataloader_num_workers=1, dataloader_prefetch_factor=None, past_index=-1, run_name='/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713', disable_tqdm=False, remove_unused_columns=False, label_names=None, load_best_model_at_end=False, metric_for_best_model='loss', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, fsdp_transformer_layer_cls_to_wrap=None, accelerator_config=AcceleratorConfig(split_batches=False, dispatch_batches=False, even_batches=True, use_seedable_sampler=True, non_blocking=False, gradient_accumulation_kwargs=None, use_configured_state=False), deepspeed=None, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=['tensorboard'], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=False, resume_from_checkpoint=None, hub_model_id=None, hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=False, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, include_for_metrics=[], eval_do_concat_batches=True, fp16_backend='auto', evaluation_strategy=None, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, dispatch_batches=None, split_batches=None, include_tokens_per_second=False, include_num_input_tokens_seen=False, neftune_noise_alpha=None, optim_target_modules=None, batch_eval_metrics=False, eval_on_start=False, use_liger_kernel=False, eval_use_gather_object=False, average_tokens_across_devices=False, sortish_sampler=False, predict_with_generate=False, generation_max_length=None, generation_num_beams=None, generation_config=GenerationConfig {\n  \"bos_token_id\": 1,\n  \"eos_token_id\": 32000,\n  \"max_new_tokens\": 2048,\n  \"pad_token_id\": 32000\n}\n, acc_strategy='token', loss_name=None, additional_saved_files=[], train_sampler_random=True, metric_warmup_step=0, train_dataset_sample=-1)"
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/trainer_state.json

@@ -0,0 +1,479 @@
+{
+  "best_metric": 0.6316511,
+  "best_model_checkpoint": "/home/pzcmoe/home/zhc/llama/output/phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200",
+  "epoch": 0.025674764915433742,
+  "eval_steps": 50,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "acc": 0.77952504,
+      "epoch": 0.0001283738245771687,
+      "grad_norm": 0.15511861443519592,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 0.82634771,
+      "memory(GiB)": 11.42,
+      "step": 1,
+      "train_speed(iter/s)": 0.092016
+    },
+    {
+      "acc": 0.77674872,
+      "epoch": 0.0006418691228858435,
+      "grad_norm": 0.14591480791568756,
+      "learning_rate": 1e-05,
+      "loss": 0.8159712,
+      "memory(GiB)": 12.15,
+      "step": 5,
+      "train_speed(iter/s)": 0.10552
+    },
+    {
+      "acc": 0.77379012,
+      "epoch": 0.001283738245771687,
+      "grad_norm": 0.151850163936615,
+      "learning_rate": 2e-05,
+      "loss": 0.8015564,
+      "memory(GiB)": 12.15,
+      "step": 10,
+      "train_speed(iter/s)": 0.107013
+    },
+    {
+      "acc": 0.7784133,
+      "epoch": 0.0019256073686575307,
+      "grad_norm": 0.15997682511806488,
+      "learning_rate": 3e-05,
+      "loss": 0.79284344,
+      "memory(GiB)": 12.15,
+      "step": 15,
+      "train_speed(iter/s)": 0.110995
+    },
+    {
+      "acc": 0.7815052,
+      "epoch": 0.002567476491543374,
+      "grad_norm": 0.14427673816680908,
+      "learning_rate": 4e-05,
+      "loss": 0.78122258,
+      "memory(GiB)": 12.9,
+      "step": 20,
+      "train_speed(iter/s)": 0.111462
+    },
+    {
+      "acc": 0.7915082,
+      "epoch": 0.0032093456144292177,
+      "grad_norm": 0.1590222716331482,
+      "learning_rate": 5e-05,
+      "loss": 0.7404335,
+      "memory(GiB)": 12.9,
+      "step": 25,
+      "train_speed(iter/s)": 0.112268
+    },
+    {
+      "acc": 0.80291214,
+      "epoch": 0.0038512147373150614,
+      "grad_norm": 0.13638992607593536,
+      "learning_rate": 6e-05,
+      "loss": 0.68063788,
+      "memory(GiB)": 12.9,
+      "step": 30,
+      "train_speed(iter/s)": 0.111517
+    },
+    {
+      "acc": 0.79923964,
+      "epoch": 0.0044930838602009054,
+      "grad_norm": 0.1271245777606964,
+      "learning_rate": 7e-05,
+      "loss": 0.68800621,
+      "memory(GiB)": 12.9,
+      "step": 35,
+      "train_speed(iter/s)": 0.110574
+    },
+    {
+      "acc": 0.80240812,
+      "epoch": 0.005134952983086748,
+      "grad_norm": 0.11937133222818375,
+      "learning_rate": 8e-05,
+      "loss": 0.69939818,
+      "memory(GiB)": 12.9,
+      "step": 40,
+      "train_speed(iter/s)": 0.109469
+    },
+    {
+      "acc": 0.78317132,
+      "epoch": 0.005776822105972592,
+      "grad_norm": 0.1665358543395996,
+      "learning_rate": 9e-05,
+      "loss": 0.75735736,
+      "memory(GiB)": 12.9,
+      "step": 45,
+      "train_speed(iter/s)": 0.108968
+    },
+    {
+      "acc": 0.80379133,
+      "epoch": 0.0064186912288584355,
+      "grad_norm": 0.14883440732955933,
+      "learning_rate": 0.0001,
+      "loss": 0.6824338,
+      "memory(GiB)": 12.9,
+      "step": 50,
+      "train_speed(iter/s)": 0.109163
+    },
+    {
+      "epoch": 0.0064186912288584355,
+      "eval_acc": 0.8081346555994846,
+      "eval_loss": 0.6729312539100647,
+      "eval_runtime": 195.4613,
+      "eval_samples_per_second": 6.436,
+      "eval_steps_per_second": 6.436,
+      "step": 50
+    },
+    {
+      "acc": 0.7984889,
+      "epoch": 0.007060560351744279,
+      "grad_norm": 0.16687534749507904,
+      "learning_rate": 9.999316524962345e-05,
+      "loss": 0.7141089,
+      "memory(GiB)": 12.9,
+      "step": 55,
+      "train_speed(iter/s)": 0.078646
+    },
+    {
+      "acc": 0.79216175,
+      "epoch": 0.007702429474630123,
+      "grad_norm": 0.17546051740646362,
+      "learning_rate": 9.997266286704631e-05,
+      "loss": 0.72419305,
+      "memory(GiB)": 12.9,
+      "step": 60,
+      "train_speed(iter/s)": 0.080466
+    },
+    {
+      "acc": 0.80358973,
+      "epoch": 0.008344298597515967,
+      "grad_norm": 0.16079305112361908,
+      "learning_rate": 9.993849845741524e-05,
+      "loss": 0.67717872,
+      "memory(GiB)": 12.9,
+      "step": 65,
+      "train_speed(iter/s)": 0.082252
+    },
+    {
+      "acc": 0.80615339,
+      "epoch": 0.008986167720401811,
+      "grad_norm": 0.1396234929561615,
+      "learning_rate": 9.989068136093873e-05,
+      "loss": 0.70638905,
+      "memory(GiB)": 12.9,
+      "step": 70,
+      "train_speed(iter/s)": 0.084
+    },
+    {
+      "acc": 0.79676814,
+      "epoch": 0.009628036843287653,
+      "grad_norm": 0.1340462565422058,
+      "learning_rate": 9.98292246503335e-05,
+      "loss": 0.71420946,
+      "memory(GiB)": 12.9,
+      "step": 75,
+      "train_speed(iter/s)": 0.08542
+    },
+    {
+      "acc": 0.79325166,
+      "epoch": 0.010269905966173496,
+      "grad_norm": 0.13259001076221466,
+      "learning_rate": 9.975414512725057e-05,
+      "loss": 0.71777782,
+      "memory(GiB)": 12.9,
+      "step": 80,
+      "train_speed(iter/s)": 0.086655
+    },
+    {
+      "acc": 0.81705523,
+      "epoch": 0.01091177508905934,
+      "grad_norm": 0.15833652019500732,
+      "learning_rate": 9.966546331768191e-05,
+      "loss": 0.62526927,
+      "memory(GiB)": 12.9,
+      "step": 85,
+      "train_speed(iter/s)": 0.088014
+    },
+    {
+      "acc": 0.80851898,
+      "epoch": 0.011553644211945184,
+      "grad_norm": 0.1456330418586731,
+      "learning_rate": 9.956320346634876e-05,
+      "loss": 0.68010745,
+      "memory(GiB)": 12.9,
+      "step": 90,
+      "train_speed(iter/s)": 0.088941
+    },
+    {
+      "acc": 0.81903477,
+      "epoch": 0.012195513334831027,
+      "grad_norm": 0.14285945892333984,
+      "learning_rate": 9.944739353007344e-05,
+      "loss": 0.6090106,
+      "memory(GiB)": 12.9,
+      "step": 95,
+      "train_speed(iter/s)": 0.089902
+    },
+    {
+      "acc": 0.82483721,
+      "epoch": 0.012837382457716871,
+      "grad_norm": 0.1478744000196457,
+      "learning_rate": 9.931806517013612e-05,
+      "loss": 0.59807158,
+      "memory(GiB)": 12.9,
+      "step": 100,
+      "train_speed(iter/s)": 0.090663
+    },
+    {
+      "epoch": 0.012837382457716871,
+      "eval_acc": 0.813768508772143,
+      "eval_loss": 0.645352840423584,
+      "eval_runtime": 195.755,
+      "eval_samples_per_second": 6.426,
+      "eval_steps_per_second": 6.426,
+      "step": 100
+    },
+    {
+      "acc": 0.80671978,
+      "epoch": 0.013479251580602715,
+      "grad_norm": 0.15444877743721008,
+      "learning_rate": 9.917525374361912e-05,
+      "loss": 0.66537051,
+      "memory(GiB)": 12.9,
+      "step": 105,
+      "train_speed(iter/s)": 0.078062
+    },
+    {
+      "acc": 0.81962786,
+      "epoch": 0.014121120703488558,
+      "grad_norm": 0.16826578974723816,
+      "learning_rate": 9.901899829374047e-05,
+      "loss": 0.60835891,
+      "memory(GiB)": 12.9,
+      "step": 110,
+      "train_speed(iter/s)": 0.079089
+    },
+    {
+      "acc": 0.80644588,
+      "epoch": 0.014762989826374402,
+      "grad_norm": 0.1519591063261032,
+      "learning_rate": 9.884934153917997e-05,
+      "loss": 0.66763787,
+      "memory(GiB)": 12.9,
+      "step": 115,
+      "train_speed(iter/s)": 0.08012
+    },
+    {
+      "acc": 0.8235981,
+      "epoch": 0.015404858949260246,
+      "grad_norm": 0.20040030777454376,
+      "learning_rate": 9.86663298624003e-05,
+      "loss": 0.5919826,
+      "memory(GiB)": 12.9,
+      "step": 120,
+      "train_speed(iter/s)": 0.081125
+    },
+    {
+      "acc": 0.81199884,
+      "epoch": 0.01604672807214609,
+      "grad_norm": 0.1842738389968872,
+      "learning_rate": 9.847001329696653e-05,
+      "loss": 0.63266978,
+      "memory(GiB)": 12.9,
+      "step": 125,
+      "train_speed(iter/s)": 0.082093
+    },
+    {
+      "acc": 0.81563005,
+      "epoch": 0.016688597195031935,
+      "grad_norm": 0.15375380218029022,
+      "learning_rate": 9.826044551386744e-05,
+      "loss": 0.61709371,
+      "memory(GiB)": 12.9,
+      "step": 130,
+      "train_speed(iter/s)": 0.082937
+    },
+    {
+      "acc": 0.80918331,
+      "epoch": 0.017330466317917778,
+      "grad_norm": 0.14841115474700928,
+      "learning_rate": 9.803768380684242e-05,
+      "loss": 0.63595695,
+      "memory(GiB)": 12.9,
+      "step": 135,
+      "train_speed(iter/s)": 0.083738
+    },
+    {
+      "acc": 0.81229877,
+      "epoch": 0.017972335440803622,
+      "grad_norm": 0.16627077758312225,
+      "learning_rate": 9.780178907671789e-05,
+      "loss": 0.64757662,
+      "memory(GiB)": 12.9,
+      "step": 140,
+      "train_speed(iter/s)": 0.084642
+    },
+    {
+      "acc": 0.804142,
+      "epoch": 0.018614204563689465,
+      "grad_norm": 0.1714244931936264,
+      "learning_rate": 9.755282581475769e-05,
+      "loss": 0.6832706,
+      "memory(GiB)": 12.9,
+      "step": 145,
+      "train_speed(iter/s)": 0.085406
+    },
+    {
+      "acc": 0.79688311,
+      "epoch": 0.019256073686575306,
+      "grad_norm": 0.18916593492031097,
+      "learning_rate": 9.729086208503174e-05,
+      "loss": 0.70668921,
+      "memory(GiB)": 12.9,
+      "step": 150,
+      "train_speed(iter/s)": 0.086183
+    },
+    {
+      "epoch": 0.019256073686575306,
+      "eval_acc": 0.8153510047395746,
+      "eval_loss": 0.6350318193435669,
+      "eval_runtime": 196.0265,
+      "eval_samples_per_second": 6.417,
+      "eval_steps_per_second": 6.417,
+      "step": 150
+    },
+    {
+      "acc": 0.80884809,
+      "epoch": 0.01989794280946115,
+      "grad_norm": 0.17633555829524994,
+      "learning_rate": 9.701596950580806e-05,
+      "loss": 0.63387117,
+      "memory(GiB)": 12.9,
+      "step": 155,
+      "train_speed(iter/s)": 0.078254
+    },
+    {
+      "acc": 0.81135025,
+      "epoch": 0.020539811932346993,
+      "grad_norm": 0.18200528621673584,
+      "learning_rate": 9.672822322997305e-05,
+      "loss": 0.65075536,
+      "memory(GiB)": 12.9,
+      "step": 160,
+      "train_speed(iter/s)": 0.078985
+    },
+    {
+      "acc": 0.82410259,
+      "epoch": 0.021181681055232836,
+      "grad_norm": 0.14853200316429138,
+      "learning_rate": 9.642770192448536e-05,
+      "loss": 0.60177903,
+      "memory(GiB)": 12.9,
+      "step": 165,
+      "train_speed(iter/s)": 0.079746
+    },
+    {
+      "acc": 0.80189457,
+      "epoch": 0.02182355017811868,
+      "grad_norm": 0.17732569575309753,
+      "learning_rate": 9.611448774886924e-05,
+      "loss": 0.68190188,
+      "memory(GiB)": 12.9,
+      "step": 170,
+      "train_speed(iter/s)": 0.080445
+    },
+    {
+      "acc": 0.81763601,
+      "epoch": 0.022465419301004524,
+      "grad_norm": 0.3583473563194275,
+      "learning_rate": 9.578866633275288e-05,
+      "loss": 0.62211475,
+      "memory(GiB)": 12.9,
+      "step": 175,
+      "train_speed(iter/s)": 0.081173
+    },
+    {
+      "acc": 0.81451321,
+      "epoch": 0.023107288423890367,
+      "grad_norm": 0.19369152188301086,
+      "learning_rate": 9.545032675245813e-05,
+      "loss": 0.63143706,
+      "memory(GiB)": 12.9,
+      "step": 180,
+      "train_speed(iter/s)": 0.081877
+    },
+    {
+      "acc": 0.809447,
+      "epoch": 0.02374915754677621,
+      "grad_norm": 0.16049319505691528,
+      "learning_rate": 9.509956150664796e-05,
+      "loss": 0.6451148,
+      "memory(GiB)": 12.9,
+      "step": 185,
+      "train_speed(iter/s)": 0.082533
+    },
+    {
+      "acc": 0.80372143,
+      "epoch": 0.024391026669662055,
+      "grad_norm": 0.19566483795642853,
+      "learning_rate": 9.473646649103818e-05,
+      "loss": 0.65715637,
+      "memory(GiB)": 12.9,
+      "step": 190,
+      "train_speed(iter/s)": 0.083203
+    },
+    {
+      "acc": 0.81516018,
+      "epoch": 0.0250328957925479,
+      "grad_norm": 0.1836969554424286,
+      "learning_rate": 9.43611409721806e-05,
+      "loss": 0.63388405,
+      "memory(GiB)": 12.9,
+      "step": 195,
+      "train_speed(iter/s)": 0.083851
+    },
+    {
+      "acc": 0.80683355,
+      "epoch": 0.025674764915433742,
+      "grad_norm": 0.15840834379196167,
+      "learning_rate": 9.397368756032445e-05,
+      "loss": 0.65765018,
+      "memory(GiB)": 12.9,
+      "step": 200,
+      "train_speed(iter/s)": 0.084362
+    },
+    {
+      "epoch": 0.025674764915433742,
+      "eval_acc": 0.8154263931167957,
+      "eval_loss": 0.6316511034965515,
+      "eval_runtime": 194.3299,
+      "eval_samples_per_second": 6.474,
+      "eval_steps_per_second": 6.474,
+      "step": 200
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.75762245139415e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5ca841f7bd4a855ee575421abc61a664c4a5bac12d5a654d3adfe5e4f1cc45d
+size 7480

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-250/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

phi3_5-vision-instruct/v6-20241130-010713/checkpoint-250/adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/home/pzcmoe/home/zhc/llama/models/Phi-3.5-vision-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "^(model.layers|model.vision_embed_tokens.img_projection)(?!.*(lm_head|output|emb|wte|shared)).*",
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}