Commit ·
c565789
1
Parent(s): 565efd1
Update checkpoint documentation
Browse files
README.md
CHANGED
|
@@ -25,12 +25,14 @@ Project page: https://yanzuo.lu/raven
|
|
| 25 |
|
| 26 |
RAVEN is a causal autoregressive text-to-video generation model built on Wan2.1-T2V-1.3B. It is designed for real-time streaming video generation by extrapolating future video chunks from previously generated content.
|
| 27 |
|
| 28 |
-
The release contains
|
| 29 |
|
| 30 |
| File | Description |
|
| 31 |
| --- | --- |
|
| 32 |
-
| `raven_model.pt` |
|
| 33 |
-
| `
|
|
|
|
|
|
|
| 34 |
|
| 35 |
RAVEN trains a causal video generator using a training-time test framework that repacks each self rollout into an interleaved sequence of clean historical endpoints and noisy denoising states. This aligns the model's training attention pattern with inference-time autoregressive extrapolation and allows downstream chunk losses to supervise the historical representations used for future predictions.
|
| 36 |
|
|
@@ -74,10 +76,51 @@ source venv/bin/activate
|
|
| 74 |
Download this model repository:
|
| 75 |
|
| 76 |
```bash
|
| 77 |
-
hf download
|
| 78 |
```
|
| 79 |
|
| 80 |
-
Then point the relevant config files to the downloaded checkpoints (`raven_model.pt`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
Reference configs:
|
| 83 |
|
|
|
|
| 25 |
|
| 26 |
RAVEN is a causal autoregressive text-to-video generation model built on Wan2.1-T2V-1.3B. It is designed for real-time streaming video generation by extrapolating future video chunks from previously generated content.
|
| 27 |
|
| 28 |
+
The release contains the RAVEN checkpoint plus three interchangeable CM-GRPO variants:
|
| 29 |
|
| 30 |
| File | Description |
|
| 31 |
| --- | --- |
|
| 32 |
+
| `raven_model.pt` | Full RAVEN backbone for causal autoregressive text-to-video generation. |
|
| 33 |
+
| `cmgrpo_raven_lora.safetensors` | CM-GRPO LoRA adapter only. Load `raven_model.pt` as the base weight and this file through the LoRA path. |
|
| 34 |
+
| `cmgrpo_raven_full.pt` | RAVEN base and CM-GRPO LoRA adapter packed into one PEFT-wrapped state dict. Load this file through the LoRA path without a separate base weight. |
|
| 35 |
+
| `cmgrpo_raven_merge.pt` | Full CM-GRPO backbone with the adapter already merged into RAVEN. Load this file as the base weight, with no LoRA block. |
|
| 36 |
|
| 37 |
RAVEN trains a causal video generator using a training-time test framework that repacks each self rollout into an interleaved sequence of clean historical endpoints and noisy denoising states. This aligns the model's training attention pattern with inference-time autoregressive extrapolation and allows downstream chunk losses to supervise the historical representations used for future predictions.
|
| 38 |
|
|
|
|
| 76 |
Download this model repository:
|
| 77 |
|
| 78 |
```bash
|
| 79 |
+
hf download mvp-lab/RAVEN --local-dir /path/to/RAVEN-weights
|
| 80 |
```
|
| 81 |
|
| 82 |
+
Then point the relevant config files to the downloaded checkpoints. RAVEN itself (`raven_model.pt`) is a single full backbone:
|
| 83 |
+
|
| 84 |
+
```jsonc
|
| 85 |
+
"backbone": {
|
| 86 |
+
"weight": "/path/to/RAVEN-weights/raven_model.pt"
|
| 87 |
+
}
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
CM-GRPO can be loaded in any of three equivalent forms:
|
| 91 |
+
|
| 92 |
+
Adapter only (`cmgrpo_raven_lora.safetensors`):
|
| 93 |
+
|
| 94 |
+
```jsonc
|
| 95 |
+
"backbone": {
|
| 96 |
+
"weight": "/path/to/RAVEN-weights/raven_model.pt",
|
| 97 |
+
"lora": {
|
| 98 |
+
"enabled": true,
|
| 99 |
+
"weight": "/path/to/RAVEN-weights/cmgrpo_raven_lora.safetensors"
|
| 100 |
+
}
|
| 101 |
+
}
|
| 102 |
+
```
|
| 103 |
+
|
| 104 |
+
Base + LoRA bundle (`cmgrpo_raven_full.pt`):
|
| 105 |
+
|
| 106 |
+
```jsonc
|
| 107 |
+
"backbone": {
|
| 108 |
+
"lora": {
|
| 109 |
+
"enabled": true,
|
| 110 |
+
"weight": "/path/to/RAVEN-weights/cmgrpo_raven_full.pt"
|
| 111 |
+
}
|
| 112 |
+
}
|
| 113 |
+
```
|
| 114 |
+
|
| 115 |
+
Merged backbone (`cmgrpo_raven_merge.pt`):
|
| 116 |
+
|
| 117 |
+
```jsonc
|
| 118 |
+
"backbone": {
|
| 119 |
+
"weight": "/path/to/RAVEN-weights/cmgrpo_raven_merge.pt"
|
| 120 |
+
}
|
| 121 |
+
```
|
| 122 |
+
|
| 123 |
+
The released CM-GRPO configs use the base + LoRA bundle form by default.
|
| 124 |
|
| 125 |
Reference configs:
|
| 126 |
|