Strip base-model identity from README and training_args.json (kept only in adapter_config.json where PEFT requires it)
Browse files- README.md +13 -11
- training_args.json +1 -4
README.md
CHANGED
|
@@ -1,5 +1,4 @@
|
|
| 1 |
---
|
| 2 |
-
base_model: Qwen/Qwen3.5-9B
|
| 3 |
library_name: peft
|
| 4 |
pipeline_tag: text-generation
|
| 5 |
tags:
|
|
@@ -10,17 +9,20 @@ tags:
|
|
| 10 |
- anonymous-release
|
| 11 |
---
|
| 12 |
|
| 13 |
-
# Anonymous Release — Judge LoRA Adapter
|
| 14 |
|
| 15 |
-
A LoRA adapter
|
| 16 |
-
|
| 17 |
-
|
| 18 |
[`anonymouscla/physground`](https://huggingface.co/datasets/anonymouscla/physground).
|
| 19 |
|
|
|
|
|
|
|
|
|
|
| 20 |
## Files
|
| 21 |
|
| 22 |
```
|
| 23 |
-
adapter_config.json # PEFT/LoRA config
|
| 24 |
adapter_model.safetensors # LoRA weights (~167 MB)
|
| 25 |
additional_config.json # ms-swift extras (lora_dtype / lr ratios)
|
| 26 |
training_args.json # sanitized training hyperparameters
|
|
@@ -30,9 +32,8 @@ training_args.json # sanitized training hyperparameters
|
|
| 30 |
|
| 31 |
| Item | Value |
|
| 32 |
| --- | --- |
|
| 33 |
-
| Base model | `Qwen/Qwen3.5-9B` |
|
| 34 |
| Tuning method | LoRA via PEFT (rank 32, α 64, dropout 0.05) |
|
| 35 |
-
| Target modules | All linear layers in the language
|
| 36 |
| Precision | bf16 with gradient checkpointing |
|
| 37 |
| Optimizer | AdamW (fused), lr = 1e-4, cosine schedule, warmup 5% |
|
| 38 |
| Batch | 1 × 8 grad-accum × 4 GPUs (global batch 32) |
|
|
@@ -47,11 +48,12 @@ anonymous dataset for prompts, physical-law tags, and example videos.
|
|
| 47 |
## Usage
|
| 48 |
|
| 49 |
```python
|
|
|
|
| 50 |
from peft import PeftModel
|
| 51 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 52 |
|
| 53 |
-
base_id = "Qwen/Qwen3.5-9B"
|
| 54 |
adapter_dir = "." # this directory
|
|
|
|
| 55 |
|
| 56 |
tokenizer = AutoTokenizer.from_pretrained(base_id)
|
| 57 |
base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="bfloat16", device_map="auto")
|
|
@@ -59,8 +61,8 @@ model = PeftModel.from_pretrained(base, adapter_dir)
|
|
| 59 |
model.eval()
|
| 60 |
```
|
| 61 |
|
| 62 |
-
The adapter expects the
|
| 63 |
-
|
| 64 |
candidate video frame/caption. Greedy decoding (`temperature = 0`) with
|
| 65 |
`max_new_tokens = 64` matches the training-time generation config.
|
| 66 |
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
library_name: peft
|
| 3 |
pipeline_tag: text-generation
|
| 4 |
tags:
|
|
|
|
| 9 |
- anonymous-release
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Anonymous Release — Judge LoRA Adapter
|
| 13 |
|
| 14 |
+
A LoRA adapter trained as a judge model that scores generated videos against
|
| 15 |
+
physical-law sub-rubrics derived from text prompts. Released anonymously
|
| 16 |
+
alongside the companion dataset
|
| 17 |
[`anonymouscla/physground`](https://huggingface.co/datasets/anonymouscla/physground).
|
| 18 |
|
| 19 |
+
The base model identifier required to load this adapter is recorded in
|
| 20 |
+
`adapter_config.json` (`base_model_name_or_path`).
|
| 21 |
+
|
| 22 |
## Files
|
| 23 |
|
| 24 |
```
|
| 25 |
+
adapter_config.json # PEFT/LoRA config
|
| 26 |
adapter_model.safetensors # LoRA weights (~167 MB)
|
| 27 |
additional_config.json # ms-swift extras (lora_dtype / lr ratios)
|
| 28 |
training_args.json # sanitized training hyperparameters
|
|
|
|
| 32 |
|
| 33 |
| Item | Value |
|
| 34 |
| --- | --- |
|
|
|
|
| 35 |
| Tuning method | LoRA via PEFT (rank 32, α 64, dropout 0.05) |
|
| 36 |
+
| Target modules | All linear layers in the language tower (vision encoder frozen) |
|
| 37 |
| Precision | bf16 with gradient checkpointing |
|
| 38 |
| Optimizer | AdamW (fused), lr = 1e-4, cosine schedule, warmup 5% |
|
| 39 |
| Batch | 1 × 8 grad-accum × 4 GPUs (global batch 32) |
|
|
|
|
| 48 |
## Usage
|
| 49 |
|
| 50 |
```python
|
| 51 |
+
import json
|
| 52 |
from peft import PeftModel
|
| 53 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 54 |
|
|
|
|
| 55 |
adapter_dir = "." # this directory
|
| 56 |
+
base_id = json.load(open(f"{adapter_dir}/adapter_config.json"))["base_model_name_or_path"]
|
| 57 |
|
| 58 |
tokenizer = AutoTokenizer.from_pretrained(base_id)
|
| 59 |
base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="bfloat16", device_map="auto")
|
|
|
|
| 61 |
model.eval()
|
| 62 |
```
|
| 63 |
|
| 64 |
+
The adapter expects the base model's default chat template, with a prompt
|
| 65 |
+
that asks the judge to answer one or more sub-rubric questions about a
|
| 66 |
candidate video frame/caption. Greedy decoding (`temperature = 0`) with
|
| 67 |
`max_new_tokens = 64` matches the training-time generation config.
|
| 68 |
|
training_args.json
CHANGED
|
@@ -1,8 +1,5 @@
|
|
| 1 |
{
|
| 2 |
-
"_comment": "Sanitized excerpt of the training configuration. Local paths
|
| 3 |
-
"base_model": "Qwen/Qwen3.5-9B",
|
| 4 |
-
"model_type": "qwen3_5",
|
| 5 |
-
"template": "qwen3_5",
|
| 6 |
"task_type": "causal_lm",
|
| 7 |
"torch_dtype": "bfloat16",
|
| 8 |
"max_length": 8192,
|
|
|
|
| 1 |
{
|
| 2 |
+
"_comment": "Sanitized excerpt of the training configuration. Local paths, tracking IDs, and base-model identity removed (see adapter_config.json for the base model required by PEFT).",
|
|
|
|
|
|
|
|
|
|
| 3 |
"task_type": "causal_lm",
|
| 4 |
"torch_dtype": "bfloat16",
|
| 5 |
"max_length": 8192,
|