Upload 2 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/teaser.jpg filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,197 @@
 ---
 license: apache-2.0
----
+language:
+  - en
+  - zh
+tags:
+  - world-model
+  - video-generation
+  - action-conditioned
+  - game-world-model
+  - first-person-shooter
+  - diffusion
+  - transformer
+  - wan2.2
+library_name: diffsynth
+pipeline_tag: image-to-video
+base_model:
+  - Wan-AI/Wan2.2-TI2V-5B
+---
+
+# <span style="color:rgb(30,100,200)">S</span><span style="color:rgb(60,160,220)">C</span><span style="color:rgb(60,160,220)">O</span><span style="color:rgb(210,120,40)">P</span><span style="color:rgb(180,80,30)">E</span>: Simulating Cross-game Operations in Playable Environments for FPS World Models
+
+<div align="center">
+  <img src="assets/teaser.jpg" alt="SCOPE Teaser" width="90%">
+
+  <p><i><b><span style="color:rgb(30,100,200)">S</span><span style="color:rgb(60,160,220)">C</span><span style="color:rgb(60,160,220)">O</span><span style="color:rgb(210,120,40)">P</span><span style="color:rgb(180,80,30)">E</span></b> is an interactive world model for FPS games with 10-DoF action control, trained on 69K clips across 7 games.</i></p>
+
+  [![Project Page](https://img.shields.io/badge/Project%20Page-SCOPE-blue)](https://github.com/z2tong/SCOPE)
+  [![GitHub](https://img.shields.io/badge/GitHub-Code-black?logo=github)](https://github.com/z2tong/SCOPE)
+  [![arXiv](https://img.shields.io/badge/arXiv-Paper-red?logo=arxiv)](https://arxiv.org/abs/XXXX.XXXXX)
+  [![License](https://img.shields.io/badge/License-Apache%202.0-green)](LICENSE)
+
+</div>
+
+## Highlights
+
+- **Hybrid Action Space** — First FPS world model to handle both continuous (4D joystick) and discrete (6 binary buttons) actions simultaneously in a unified framework
+- **Dense High-Frequency Control** — Resolves overlapping control signals at every frame, unlike prior methods limited to sparse inputs
+- **Action Composition** — Supports simultaneous multi-action combinations (e.g., moving + aiming + firing), reflecting real gameplay complexity
+- **Cross-Game Generalization** — A single world model trained on 7 diverse FPS games that generalizes zero-shot to unseen game environments
+- **In-Scope / Out-of-Scope Decoupling** — Spatially selective conditioning that separates localized in-scope effects (weapon, HUD) from stable out-of-scope world generation, without segmentation labels
+
+## Model Overview
+
+SCOPE is an interactive world model for first-person shooter (FPS) games. Unlike prior game world models that only handle sparse, single-modality actions, SCOPE processes **hybrid action inputs** (continuous joystick + discrete buttons) at **dense, high-frequency** rates, supporting real-time **action composition** — multiple simultaneous inputs such as moving, aiming, and firing in the same frame.
+
+Built on [Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B), SCOPE inserts a conditioning module into each transformer block that reshapes features into per-pixel temporal sequences. Each spatial position computes its action response from local visual content, naturally separating in-scope effects (e.g., weapon firing, reloading) from out-of-scope world generation (e.g., stable surroundings) — without any segmentation labels. Trained on [**CrossFPS**](https://huggingface.co/collections/zizhaotong/crossfps) (69K clips, 7 games, 10-DoF), SCOPE learns general visual-to-action mappings rather than game-specific patterns, enabling zero-shot transfer to unseen scenes.
+
+### Architecture
+
+| Component | Details |
+|-----------|---------|
+| **Base Model** | Wan2.2-TI2V-5B (DiT, 30 layers) |
+| **Action Module** | Per-block conditioning, per-pixel temporal sequences |
+| **Text Encoder** | UMT5-XXL |
+| **VAE** | Wan2.2 Video VAE (temporal compression 4x, spatial compression 8x) |
+| **Parameters** | ~5B total (1575 tensors) |
+| **Precision** | BFloat16 |
+
+### Generation Specs
+
+| Property | Value |
+|----------|-------|
+| Resolution | 480 x 832 |
+| Frame Count | 81 frames |
+| Frame Rate | 20 FPS |
+| Duration | ~4 seconds |
+| Inference Steps | 30 (default) |
+
+## Action Input Format
+
+SCOPE accepts 10-DoF action inputs per frame via a Parquet file:
+
+**Controller Buttons (6D binary):**
+
+| Index | Column | Action |
+|:-----:|--------|--------|
+| 0 | `right_trigger` | Fire (RT) |
+| 1 | `left_trigger` | Aim Down Sights (LT) |
+| 2 | `south` | Jump (A) |
+| 3 | `right_thumb` | Melee (R3) |
+| 4 | `west` | Reload (X) |
+| 5 | `north` | Weapon Switch (Y) |
+
+**Dual Joystick (4D continuous):**
+
+| Column | Axes | Function |
+|--------|------|----------|
+| `j_left` | [x, y] | Character movement (left stick) |
+| `j_right` | [x, y] | Camera rotation (right stick) |
+
+## Quick Start
+
+### Requirements
+
+- Python >= 3.10
+- PyTorch >= 2.5 with CUDA support
+- GPU: NVIDIA with >= 24GB VRAM (single GPU inference with CPU offload)
+- [DiffSynth](https://github.com/modelscope/DiffSynth-Studio) framework
+
+### Installation
+
+```bash
+git clone https://github.com/z2tong/SCOPE.git
+cd SCOPE
+pip install -e .
+```
+
+### Download Weights
+
+```bash
+# Download all weights (SCOPE DiT + Text Encoder + VAE + Tokenizer) in one command
+huggingface-cli download z2tong/SCOPE --local-dir ./SCOPE
+```
+
+The model directory contains everything needed for inference:
+
+```
+SCOPE/
+├── model-00001-of-00003.safetensors   # SCOPE DiT shard 1
+├── model-00002-of-00003.safetensors   # SCOPE DiT shard 2
+├── model-00003-of-00003.safetensors   # SCOPE DiT shard 3
+├── model.safetensors.index.json       # Shard index
+├── models_t5_umt5-xxl-enc-bf16.pth   # Text Encoder (UMT5-XXL)
+├── Wan2.2_VAE.pth                     # Video VAE
+├── google/umt5-xxl/                   # Tokenizer
+└── config.json                        # Model config
+```
+
+### Inference
+
+**Single image + action sequence:**
+
+```bash
+python inference.py \
+    --model_dir ./SCOPE \
+    --input_image input.png \
+    --action_path action.parquet \
+    --prompt "First-person shooter perspective in a modern city" \
+    --seed 42
+```
+
+**Batch processing (directory of images):**
+
+```bash
+python inference.py \
+    --model_dir ./SCOPE \
+    --input_image_dir ./images \
+    --action_path action.parquet \
+    --prompt "First-person view in a battlefield" \
+    --output_dir ./outputs
+```
+
+For full usage details and advanced options, see the [GitHub repository](https://github.com/z2tong/SCOPE).
+
+## Repository Contents
+
+This repo contains **all weights** needed for inference in a single download:
+
+| File | Component | Size |
+|------|-----------|------|
+| `model-00001-of-00003.safetensors` | SCOPE DiT shard 1 | ~5.0 GB |
+| `model-00002-of-00003.safetensors` | SCOPE DiT shard 2 | ~5.0 GB |
+| `model-00003-of-00003.safetensors` | SCOPE DiT shard 3 | ~4.6 GB |
+| `model.safetensors.index.json` | Shard index mapping | — |
+| `models_t5_umt5-xxl-enc-bf16.pth` | Text Encoder (UMT5-XXL) | ~20 GB |
+| `Wan2.2_VAE.pth` | Video VAE | ~700 MB |
+| `google/umt5-xxl/` | Tokenizer | ~10 MB |
+| `config.json` | Model architecture config | — |
+
+> **Inference code** is available at [github.com/z2tong/SCOPE](https://github.com/z2tong/SCOPE).
+
+## CrossFPS Dataset
+
+SCOPE is trained on [**CrossFPS**](https://huggingface.co/collections/zizhaotong/crossfps), the first multi-game FPS dataset with frame-aligned action telemetry:
+
+| Property | Value |
+|----------|-------|
+| Games | 7 diverse FPS titles |
+| Total Clips | 69,000+ |
+| Action Dimensions | 10-DoF (6 buttons + 4D joystick) |
+| Annotation | Frame-aligned action telemetry |
+| Curation | Gameplay-bias removal for general visual-to-action mapping |
+
+## Citation
+
+```bibtex
+@article{scope2025,
+  title={SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models},
+  author={Tong, Zizhao and Lai, Hongfeng and Wang, Zeqing and Xing, Zhaohu and Cheng, Kexu and Xu, Haoran and Pu, Zhao and Zhu, Shangwen and Feng, Ruili and Zhao, Jian and Zhang, Yan and Tang, Hao and Jin, Yeying and Shao, Ling},
+  year={2025}
+}
+```
+
+## Acknowledgements
+
+We thank the [Wan Team](https://huggingface.co/Wan-AI) for open-sourcing Wan2.2 and the [DiffSynth](https://github.com/modelscope/DiffSynth-Studio) team for the inference framework.