Duplicate from tencent/HY-World-2.0

Co-authored-by: TencentOpen <TencentOpen@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,53 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/interactive.gif filter=lfs diff=lfs merge=lfs -text
+assets/mesh_en.gif filter=lfs diff=lfs merge=lfs -text
+assets/mesh.gif filter=lfs diff=lfs merge=lfs -text
+assets/overview.png filter=lfs diff=lfs merge=lfs -text
+assets/recon.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_1.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_10.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_2.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_3.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_4.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_5.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_6.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_7.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_8.gif filter=lfs diff=lfs merge=lfs -text
+assets/screenshot_9.gif filter=lfs diff=lfs merge=lfs -text
+assets/teaser.png filter=lfs diff=lfs merge=lfs -text
+assets/hyworld2_en.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/recon_en.gif filter=lfs diff=lfs merge=lfs -text

DOCUMENTATION.md

@@ -0,0 +1,458 @@
+# HunyuanWorld 2.0 — Documentation
+This document provides detailed usage guides, parameter references, and output format specifications for each component of HunyuanWorld 2.0.
+
+## Table of Contents
+- [WorldMirror 2.0 (World Reconstruction)](#worldmirror-20-world-reconstruction)
+  - [Overview](#overview)
+  - [Python API](#python-api)
+    - [`WorldMirrorPipeline.from_pretrained`](#worldmirrorpipelinefrom_pretrained)
+    - [`WorldMirrorPipeline.__call__`](#worldmirrorpipelinecall)
+  - [CLI Reference](#cli-reference)
+  - [Output Format](#output-format)
+    - [File Structure](#file-structure)
+    - [Prediction Dictionary](#prediction-dictionary)
+  - [Prior Injection](#prior-injection)
+    - [Camera Parameters (JSON)](#camera-parameters-json)
+    - [Depth Maps (Folder)](#depth-maps-folder)
+    - [Combining Priors](#combining-priors)
+  - [Multi-GPU Inference](#multi-gpu-inference)
+  - [Advanced Options](#advanced-options)
+    - [Disabling Prediction Heads](#disabling-prediction-heads)
+    - [Mask Filtering](#mask-filtering)
+    - [Point Cloud Compression](#point-cloud-compression)
+  - [Gradio App](#gradio-app)
+- [Panorama Generation](#panorama-generation)
+- [World Generation](#world-generation)
+
+---
+## WorldMirror 2.0 (World Reconstruction)
+### Overview
+WorldMirror 2.0 is a unified feed-forward model for comprehensive 3D geometric prediction from multi-view images or video. It simultaneously generates:
+- **3D point clouds** in world coordinates
+- **Per-view depth maps** in camera frame
+- **Surface normals** in camera coordinates
+- **Camera poses** (c2w) and **intrinsics**
+- **3D Gaussian Splatting** attributes (means, scales, rotations, opacities, SH coefficients)
+
+Key improvements over WorldMirror 1.0:
+- **Normalized RoPE** for flexible resolution inference
+- **Depth mask prediction** for robust invalid pixel handling
+- **Sequence Parallel + FSDP + BF16** for efficient multi-GPU inference
+
+---
+### Python API
+#### `WorldMirrorPipeline.from_pretrained`
+Factory method to load the model and create a pipeline instance.
+
+```python
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+
+pipeline = WorldMirrorPipeline.from_pretrained(
+    pretrained_model_name_or_path="tencent/HY-World-2.0",
+    subfolder="HY-WorldMirror-2.0",
+    config_path=None,
+    ckpt_path=None,
+    use_fsdp=False,
+    enable_bf16=False,
+    fsdp_cpu_offload=False,
+    disable_heads=None,
+)
+```
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `pretrained_model_name_or_path` | `str` | `"tencent/HY-World-2.0"` | HuggingFace repo ID or local path |
+| `subfolder` | `str` | `"HY-WorldMirror-2.0"` | Subfolder inside the repo containing WorldMirror checkpoint (`model.safetensors` + config) |
+| `config_path` | `str` | `None` | Training config YAML (used with `ckpt_path` for custom checkpoints) |
+| `ckpt_path` | `str` | `None` | Checkpoint file (`.ckpt` / `.safetensors`). When provided with `config_path`, loads model from local checkpoint instead of HuggingFace |
+| `use_fsdp` | `bool` | `False` | Shard parameters across GPUs via Fully Sharded Data Parallel |
+| `enable_bf16` | `bool` | `False` | Use bfloat16 precision (except numerically critical layers) |
+| `fsdp_cpu_offload` | `bool` | `False` | Offload FSDP parameters to CPU (saves GPU memory at the cost of speed) |
+| `disable_heads` | `list[str]` | `None` | Heads to disable and free from memory. Options: `"camera"`, `"depth"`, `"normal"`, `"points"`, `"gs"` |
+
+**Notes:**
+- Distributed mode is auto-detected from `WORLD_SIZE` environment variable (set by `torchrun`).
+- When using multi-GPU, each rank must call `from_pretrained` — the method handles `dist.init_process_group` internally.
+
+---
+#### `WorldMirrorPipeline.__call__`
+Run inference on a set of images or a video.
+
+```python
+result = pipeline(
+    input_path,
+    output_path="inference_output",
+    **kwargs,
+)
+```
+
+Returns the output directory path (`str`), or `None` if the input was skipped.
+
+**Inference Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `input_path` | `str` | *(required)* | Directory of images or path to a video file |
+| `output_path` | `str` | `"inference_output"` | Root output directory |
+| `target_size` | `int` | `952` | Maximum inference resolution (longest edge). Images are resized + center-cropped to the nearest multiple of 14 |
+| `fps` | `int` | `1` | FPS for extracting frames from video input |
+| `video_strategy` | `str` | `"new"` | Video frame extraction strategy: `"new"` (motion-aware) or `"old"` (uniform FPS) |
+| `video_min_frames` | `int` | `1` | Minimum number of frames to extract from video |
+| `video_max_frames` | `int` | `32` | Maximum number of frames to extract from video |
+
+**Save Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `save_depth` | `bool` | `True` | Save per-view depth maps (PNG visualization + NPY raw values) |
+| `save_normal` | `bool` | `True` | Save per-view surface normal maps (PNG) |
+| `save_gs` | `bool` | `True` | Save 3D Gaussian Splatting as `gaussians.ply` |
+| `save_camera` | `bool` | `True` | Save camera parameters as `camera_params.json` |
+| `save_points` | `bool` | `True` | Save depth-derived point cloud as `points.ply` |
+| `save_colmap` | `bool` | `False` | Save COLMAP-format sparse reconstruction (`sparse/0/`) |
+| `save_conf` | `bool` | `False` | Save depth confidence maps |
+| `save_sky_mask` | `bool` | `False` | Save sky segmentation masks |
+
+**Mask Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `apply_sky_mask` | `bool` | `True` | Filter out sky regions from point clouds and Gaussians |
+| `apply_edge_mask` | `bool` | `True` | Filter out edge/discontinuity regions |
+| `apply_confidence_mask` | `bool` | `False` | Filter out low-confidence predictions |
+| `sky_mask_source` | `str` | `"auto"` | Sky mask method: `"auto"` (ONNX + model fusion), `"model"` (model predictions only), `"onnx"` (external segmentation only) |
+| `model_sky_threshold` | `float` | `0.45` | Threshold for model-based sky detection |
+| `confidence_percentile` | `float` | `10.0` | Percentile threshold for confidence filtering (bottom N% removed) |
+| `edge_normal_threshold` | `float` | `1.0` | Normal edge detection tolerance |
+| `edge_depth_threshold` | `float` | `0.03` | Depth edge detection relative tolerance |
+
+**Compression Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `compress_pts` | `bool` | `True` | Compress point clouds via voxel merging + random sampling |
+| `compress_pts_max_points` | `int` | `2,000,000` | Maximum number of points after compression |
+| `compress_pts_voxel_size` | `float` | `0.002` | Voxel size for point cloud merging |
+| `max_resolution` | `int` | `1920` | Maximum resolution for saved output images |
+| `compress_gs_max_points` | `int` | `5,000,000` | Maximum number of Gaussians after voxel pruning |
+
+**Prior Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `prior_cam_path` | `str` | `None` | Path to camera parameters JSON file |
+| `prior_depth_path` | `str` | `None` | Path to directory containing depth map files |
+
+**Rendered Video Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `save_rendered` | `bool` | `False` | Render interpolated fly-through video from Gaussian splats |
+| `render_interp_per_pair` | `int` | `15` | Number of interpolated frames between each camera pair |
+| `render_depth` | `bool` | `False` | Also render a depth visualization video |
+
+**Misc Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `log_time` | `bool` | `True` | Print timing report and save `pipeline_timing.json` |
+| `strict_output_path` | `str` | `None` | If set, save results directly to this path without `<case_name>/<timestamp>` subdirectories |
+
+---
+### CLI Reference
+All `__call__` parameters are exposed as CLI arguments:
+
+```bash
+python -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --output_path inference_output \
+    --target_size 952 \
+    --prior_cam_path path/to/camera_params.json \
+    --prior_depth_path path/to/depth_dir/ \
+```
+
+**Boolean flag conventions:**
+
+| Enable | Disable |
+|--------|---------|
+| `--save_colmap` | *(omit)* |
+| `--save_conf` | *(omit)* |
+| `--save_sky_mask` | *(omit)* |
+| `--apply_sky_mask` (default on) | `--no_sky_mask` |
+| `--apply_edge_mask` (default on) | `--no_edge_mask` |
+| `--apply_confidence_mask` | *(omit)* |
+| `--compress_pts` (default on) | `--no_compress_pts` |
+| `--log_time` (default on) | `--no_log_time` |
+| *(default on)* `save_depth` | `--no_save_depth` |
+| *(default on)* `save_normal` | `--no_save_normal` |
+| *(default on)* `save_gs` | `--no_save_gs` |
+| *(default on)* `save_camera` | `--no_save_camera` |
+| *(default on)* `save_points` | `--no_save_points` |
+| `--save_rendered` | *(omit)* |
+| `--render_depth` | *(omit)* |
+
+**Additional CLI-only arguments:**
+
+| Argument | Description |
+|----------|-------------|
+| `--config_path` | Training config YAML for custom checkpoint loading |
+| `--ckpt_path` | Local checkpoint file path |
+| `--use_fsdp` | Enable FSDP multi-GPU sharding |
+| `--enable_bf16` | Enable bfloat16 mixed precision |
+| `--fsdp_cpu_offload` | Offload FSDP params to CPU |
+| `--disable_heads` | Space-separated list of heads to disable (e.g. `--disable_heads camera normal`) |
+| `--no_interactive` | Exit after first inference (skip interactive prompt loop) |
+
+---
+### Output Format
+#### File Structure
+
+```
+inference_output/
+└── <case_name>/
+    └── <timestamp>/
+        ├── depth/
+        │   ├── depth_0000.png      # Normalized depth visualization
+        │   ├── depth_0000.npy      # Raw float32 depth values [H, W]
+        │   └── ...
+        ��── normal/
+        │   ├── normal_0000.png     # Normal map visualization (RGB)
+        │   └── ...
+        ├── camera_params.json      # Camera extrinsics & intrinsics
+        ├── gaussians.ply           # 3D Gaussian Splatting (standard format)
+        ├── points.ply              # Colored point cloud
+        ├── sparse/                 # COLMAP format (if --save_colmap)
+        │   └── 0/
+        │       ├── cameras.bin
+        │       ├── images.bin
+        │       └── points3D.bin
+        ├── rendered/               # Rendered video (if --save_rendered)
+        │   ├── rendered_rgb.mp4
+        │   └── rendered_depth.mp4  # (if --render_depth)
+        └── pipeline_timing.json    # Performance timing report
+```
+
+#### Prediction Dictionary
+When using the Python API, `pipeline(...)` internally produces a `predictions` dictionary with the following keys:
+
+```python
+# Geometry
+predictions["depth"]        # [B, S, H, W, 1]  — Z-depth in camera frame
+predictions["depth_conf"]   # [B, S, H, W]     — Depth confidence
+predictions["normals"]      # [B, S, H, W, 3]  — Surface normals in camera coords
+predictions["normals_conf"] # [B, S, H, W]     — Normal confidence
+predictions["pts3d"]        # [B, S, H, W, 3]  — 3D point maps in world coords
+predictions["pts3d_conf"]   # [B, S, H, W]     — Point cloud confidence
+# Camera
+predictions["camera_poses"] # [B, S, 4, 4]     — Camera-to-world (c2w), OpenCV convention
+predictions["camera_intrs"] # [B, S, 3, 3]     — Camera intrinsic matrices
+predictions["camera_params"]# [B, S, 9]        — Compact camera vector (translation, quaternion, fov_v, fov_u)
+# 3D Gaussian Splatting
+predictions["splats"]["means"]      # [B, N, 3] — Gaussian centers
+predictions["splats"]["scales"]     # [B, N, 3] — Gaussian scales
+predictions["splats"]["quats"]      # [B, N, 4] — Gaussian rotations (quaternions)
+predictions["splats"]["opacities"]  # [B, N]    — Gaussian opacities
+predictions["splats"]["sh"]         # [B, N, 1, 3] — Spherical harmonics (degree 0)
+predictions["splats"]["weights"]    # [B, N]    — Per-Gaussian confidence weights
+```
+
+Where `B` = batch size (always 1 for inference), `S` = number of input views, `H, W` = image dimensions, `N` = total Gaussians (`S × H × W`).
+
+---
+### Prior Injection
+WorldMirror 2.0 accepts three types of geometric priors as conditioning inputs. Priors are automatically detected from the provided files.
+
+| Prior Type | Condition | Input Format |
+|------------|-----------|--------------|
+| Camera Pose | `cond_flags[0]` | c2w 4×4 matrix (OpenCV convention) |
+| Depth Map | `cond_flags[1]` | Per-view float depth maps |
+| Intrinsics | `cond_flags[2]` | 3×3 intrinsic matrix |
+
+#### Camera Parameters (JSON)
+The camera parameter file follows the same format as the `camera_params.json` output by the pipeline:
+
+```json
+{
+  "num_cameras": 2,
+  "extrinsics": [
+    {
+      "camera_id": 0,
+      "matrix": [
+        [0.98, 0.01, -0.17, 0.52],
+        [-0.01, 0.99, 0.01, -0.03],
+        [0.17, -0.01, 0.98, 1.20],
+        [0.0, 0.0, 0.0, 1.0]
+      ]
+    }
+  ],
+  "intrinsics": [
+    {
+      "camera_id": 0,
+      "matrix": [
+        [525.0, 0.0, 320.0],
+        [0.0, 525.0, 240.0],
+        [0.0, 0.0, 1.0]
+      ]
+    }
+  ]
+}
+```
+
+**Field descriptions:**
+
+| Field | Description |
+|-------|-------------|
+| `camera_id` | Integer index (`0`, `1`, `2`, ...) or image filename stem without extension (e.g., `"image_0001"`) |
+| `extrinsics.matrix` | 4×4 camera-to-world (c2w) transformation matrix, OpenCV coordinate convention |
+| `intrinsics.matrix` | 3×3 camera intrinsic matrix in pixels (`fx, fy` = focal lengths; `cx, cy` = principal point) |
+
+**Important notes:**
+- `extrinsics` and `intrinsics` lists can be provided independently or together. An empty list `[]` or missing key means that prior is unavailable.
+- **Intrinsics resolution:** Values should correspond to the **original image resolution**. The pipeline automatically adjusts for inference-time resize + center-crop.
+- **Extrinsics alignment:** The pipeline automatically normalizes all extrinsics relative to the first view, consistent with training behavior.
+#### Depth Maps (Folder)
+Depth maps are stored as individual files in a directory. Filenames should match the input image filenames. Supported formats: `.npy`, `.exr`, `.png` (16-bit).
+
+```
+prior_depth/
+├── image_0001.npy    # float32, shape [H, W]
+├── image_0002.npy
+└── ...
+```
+
+#### Combining Priors
+Priors can be freely combined. Examples:
+
+```bash
+# Only intrinsics
+python -m hyworld2.worldrecon.pipeline --input_path images/ \
+    --prior_cam_path camera_intrinsics_only.json
+# Only depth
+python -m hyworld2.worldrecon.pipeline --input_path images/ \
+    --prior_depth_path depth_maps/
+# Camera poses + intrinsics + depth
+python -m hyworld2.worldrecon.pipeline --input_path images/ \
+    --prior_cam_path camera_params.json \
+    --prior_depth_path depth_maps/
+```
+
+---
+### Multi-GPU Inference
+WorldMirror 2.0 supports **Sequence Parallel (SP)** inference across multiple GPUs, where token sequences are sharded across ranks in the ViT backbone, and DPT heads process frames in parallel.
+
+> **Requirement:** The number of input images must be **>= the number of GPUs** (`nproc_per_node`). For example, with 8 GPUs you need at least 8 input images. The pipeline will raise an error if this condition is not met.
+
+```bash
+# 2 GPUs with FSDP + bf16
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --use_fsdp --enable_bf16
+# 4 GPUs
+torchrun --nproc_per_node=4 -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --use_fsdp --enable_bf16
+# Python API (inside a torchrun script)
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+pipeline = WorldMirrorPipeline.from_pretrained(
+    'tencent/HY-World-2.0',
+    use_fsdp=True,
+    enable_bf16=True,
+)
+pipeline('path/to/images')
+```
+
+**What happens under the hood:**
+1. `from_pretrained` auto-detects `WORLD_SIZE > 1` and initializes `torch.distributed`.
+2. The model is loaded on rank 0 and broadcast via `sync_module_states=True`.
+3. FSDP shards parameters across the SP process group.
+4. DPT prediction heads split frames across ranks and `AllGather` results.
+5. Post-processing (mask computation, saving) runs on rank 0 only.
+
+---
+### Advanced Options
+#### Disabling Prediction Heads
+To save memory when you only need specific outputs:
+
+```python
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+
+pipeline = WorldMirrorPipeline.from_pretrained(
+    'tencent/HY-World-2.0',
+    disable_heads=["normal", "points"],  # free ~200M params
+)
+```
+
+Available heads: `"camera"`, `"depth"`, `"normal"`, `"points"`, `"gs"`.
+#### Mask Filtering
+The pipeline supports three types of output filtering to improve point cloud and Gaussian quality:
+1. **Sky mask** (`apply_sky_mask=True`): Removes sky regions using an ONNX-based segmentation model, optionally fused with model-predicted depth masks.
+2. **Edge mask** (`apply_edge_mask=True`): Removes points at depth/normal discontinuities (object boundaries).
+3. **Confidence mask** (`apply_confidence_mask=False`): Removes the bottom N% of points by prediction confidence.
+These masks are applied independently to both the `points.ply` (depth-based) and `gaussians.ply` (GS-based) outputs. The GS output uses its own depth predictions for edge detection when available.
+#### Point Cloud Compression
+When `compress_pts=True` (default), the depth-derived point cloud undergoes:
+1. **Voxel merging**: Points within each voxel (size controlled by `compress_pts_voxel_size`) are merged via weighted averaging.
+2. **Random subsampling**: If the result exceeds `compress_pts_max_points`, points are uniformly subsampled.
+Similarly, Gaussians are voxel-pruned (weighted averaging of means, scales, quaternions, colors, opacities) and optionally subsampled to `compress_gs_max_points`.
+
+---
+### Gradio App
+An interactive web demo for WorldMirror 2.0. Upload images or videos and visualize 3DGS, point clouds, depth maps, normal maps, and camera parameters in your browser.
+**Quick start:**
+
+```bash
+# Single GPU
+python -m hyworld2.worldrecon.gradio_app
+
+# Multi-GPU
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.gradio_app \
+    --use_fsdp --enable_bf16
+```
+
+**With a local checkpoint:**
+
+```bash
+python -m hyworld2.worldrecon.gradio_app \
+    --config_path /path/to/config.yaml \
+    --ckpt_path /path/to/checkpoint.safetensors
+```
+
+**With a public link (e.g., for Colab or remote servers):**
+
+```bash
+python -m hyworld2.worldrecon.gradio_app --share
+```
+
+**Arguments:**
+
+| Argument | Default | Description |
+|----------|---------|-------------|
+| `--port` | `8081` | Server port |
+| `--host` | `0.0.0.0` | Server host |
+| `--share` | `False` | Create a public Gradio link |
+| `--examples_dir` | `./examples/worldrecon` | Path to example scenes directory |
+| `--config_path` | `None` | Training config YAML (used with `--ckpt_path`) |
+| `--ckpt_path` | `None` | Local checkpoint file (`.ckpt` / `.safetensors`) |
+| `--use_fsdp` | `False` | Enable FSDP multi-GPU sharding |
+| `--enable_bf16` | `False` | Enable bfloat16 mixed precision |
+| `--fsdp_cpu_offload` | `False` | Offload FSDP params to CPU (saves GPU memory) |
+
+> **Important:** In multi-GPU mode, the number of input images must be **>= the number of GPUs**.
+
+---
+## Panorama Generation
+*Coming soon.*
+This section will document the panorama generation model, including:
+- Text-to-panorama and image-to-panorama APIs
+- Model architecture (MMDiT-based implicit perspective-to-ERP mapping)
+- Configuration parameters
+- Output formats
+
+---
+## World Generation
+*Coming soon.*
+This section will document the world generation pipeline, including:
+- Trajectory planning configuration
+- World expansion with memory-driven video generation
+- World composition (point cloud expansion + 3DGS optimization)
+- End-to-end generation from text/image to navigable 3D world

DOCUMENTATION_zh.md

@@ -0,0 +1,458 @@
+# HunyuanWorld 2.0 — 文档
+本文档提供了 HunyuanWorld 2.0 各组件的详细使用指南、参数参考和输出格式说明。
+
+## 目录
+- [WorldMirror 2.0（世界重建）](#worldmirror-20世界重建)
+  - [概述](#概述)
+  - [Python API](#python-api)
+    - [`WorldMirrorPipeline.from_pretrained`](#worldmirrorpipelinefrom_pretrained)
+    - [`WorldMirrorPipeline.__call__`](#worldmirrorpipelinecall)
+  - [命令行参考](#命令行参考)
+  - [输出格式](#输出格式)
+    - [文件结构](#文件结构)
+    - [预测字典](#预测字典)
+  - [先验注入](#先验注入)
+    - [相机参数（JSON）](#相机参数json)
+    - [深度图（文件夹）](#深度图文件夹)
+    - [组合先验](#组合先验)
+  - [多卡推理](#多卡推理)
+  - [高级选项](#高级选项)
+    - [禁用预测头](#禁用预测头)
+    - [掩码过滤](#掩码过滤)
+    - [点云压缩](#点云压缩)
+  - [Gradio 应用](#gradio-应用)
+- [全景生成](#全景生成)
+- [世界生成](#世界生成)
+
+---
+## WorldMirror 2.0（世界重建）
+### 概述
+WorldMirror 2.0 是一个统一的前馈模型，用于从多视图图像或视频进行全面的3D几何预测。它能同时生成：
+- **3D 点云**（世界坐标系）
+- **逐视图深度图**（相机坐标系）
+- **表面法线**（相机坐标系）
+- **相机位姿**（c2w）和**内参**
+- **3D 高斯点云**属性（均值、尺度、旋转、不透明度、球谐系数）
+
+相比 WorldMirror 1.0 的关键改进：
+- **归一化 RoPE** 支持灵活分辨率推理
+- **深度掩码预测** 实现稳健的无效像素处理
+- **序列并行 + FSDP + BF16** 实现高效多卡推理
+
+---
+### Python API
+#### `WorldMirrorPipeline.from_pretrained`
+工厂方法，用于加载模型并创建 Pipeline 实例。
+
+```python
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+
+pipeline = WorldMirrorPipeline.from_pretrained(
+    pretrained_model_name_or_path="tencent/HY-World-2.0",
+    subfolder="HY-WorldMirror-2.0",
+    config_path=None,
+    ckpt_path=None,
+    use_fsdp=False,
+    enable_bf16=False,
+    fsdp_cpu_offload=False,
+    disable_heads=None,
+)
+```
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `pretrained_model_name_or_path` | `str` | `"tencent/HY-World-2.0"` | HuggingFace 仓库 ID 或本地路径 |
+| `subfolder` | `str` | `"HY-WorldMirror-2.0"` | 仓库内包含 WorldMirror 检查点的子文件夹（`model.safetensors` + 配置文件） |
+| `config_path` | `str` | `None` | 训练配置 YAML（与 `ckpt_path` 配合用于自定义检查点） |
+| `ckpt_path` | `str` | `None` | 检查点文件（`.ckpt` / `.safetensors`）。与 `config_path` 一起使用时，从本地检查点加载模型而非 HuggingFace |
+| `use_fsdp` | `bool` | `False` | 通过完全分片数据并行（FSDP）在多卡间分片参数 |
+| `enable_bf16` | `bool` | `False` | 使用 bfloat16 精度（数值敏感层除外） |
+| `fsdp_cpu_offload` | `bool` | `False` | 将 FSDP 参数卸载到 CPU（节省显存但降低速度） |
+| `disable_heads` | `list[str]` | `None` | 要禁用并释放内存的预测头。可选：`"camera"`、`"depth"`、`"normal"`、`"points"`、`"gs"` |
+
+**说明：**
+- 分布式模式通过 `WORLD_SIZE` 环境变量（由 `torchrun` 设置）自动检测。
+- 使用多卡时，每个 rank 都需要调用 `from_pretrained`——该方法内部处理 `dist.init_process_group`。
+
+---
+#### `WorldMirrorPipeline.__call__`
+对一组图像或视频运行推理。
+
+```python
+result = pipeline(
+    input_path,
+    output_path="inference_output",
+    **kwargs,
+)
+```
+
+返回输出目录路径（`str`），如果输入被跳过则返回 `None`。
+
+**推理参数：**
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `input_path` | `str` | *（必填）* | 图像目录或视频文件路径 |
+| `output_path` | `str` | `"inference_output"` | 输出根目录 |
+| `target_size` | `int` | `952` | 最大推理分辨率（最长边）。图像将被缩放 + 中心裁剪到最近的 14 的倍数 |
+| `fps` | `int` | `1` | 从视频输入提取帧的帧率 |
+| `video_strategy` | `str` | `"new"` | 视频帧提取策略：`"new"`（运动感知）或 `"old"`（均匀 FPS） |
+| `video_min_frames` | `int` | `1` | 从视频中提取的最少帧数 |
+| `video_max_frames` | `int` | `32` | 从视频中提取的最多帧数 |
+
+**保存参数：**
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `save_depth` | `bool` | `True` | 保存逐视图深度图（PNG 可视化 + NPY 原始值） |
+| `save_normal` | `bool` | `True` | 保存逐视图表面法线图（PNG） |
+| `save_gs` | `bool` | `True` | 保存 3D 高斯点云为 `gaussians.ply` |
+| `save_camera` | `bool` | `True` | 保存相机参数为 `camera_params.json` |
+| `save_points` | `bool` | `True` | 保存基于深度的点云为 `points.ply` |
+| `save_colmap` | `bool` | `False` | 保存 COLMAP 格式的稀疏重建（`sparse/0/`） |
+| `save_conf` | `bool` | `False` | 保存深度置信度图 |
+| `save_sky_mask` | `bool` | `False` | 保存天空分割掩码 |
+
+**掩码参数：**
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `apply_sky_mask` | `bool` | `True` | 从点云和高斯中过滤天空区域 |
+| `apply_edge_mask` | `bool` | `True` | 过滤边缘/不连续区域 |
+| `apply_confidence_mask` | `bool` | `False` | 过滤低置信度预测 |
+| `sky_mask_source` | `str` | `"auto"` | 天空掩码方法：`"auto"`（ONNX + 模型融合）、`"model"`（仅模型预测）、`"onnx"`（仅外部分割） |
+| `model_sky_threshold` | `float` | `0.45` | 基于模型的天空检测阈值 |
+| `confidence_percentile` | `float` | `10.0` | 置信度过滤的百分位阈值（移除最低 N%） |
+| `edge_normal_threshold` | `float` | `1.0` | 法线边缘检测容差 |
+| `edge_depth_threshold` | `float` | `0.03` | 深度边缘检测相对容差 |
+
+**压缩参数：**
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `compress_pts` | `bool` | `True` | 通过体素合并 + 随机采样压缩点云 |
+| `compress_pts_max_points` | `int` | `2,000,000` | 压缩后的最大点数 |
+| `compress_pts_voxel_size` | `float` | `0.002` | 点云合并的体素大小 |
+| `max_resolution` | `int` | `1920` | 保存输出图像的最大分辨率 |
+| `compress_gs_max_points` | `int` | `5,000,000` | 体素剪枝后的最大高斯数 |
+
+**先验参数：**
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `prior_cam_path` | `str` | `None` | 相机参数 JSON 文件路径 |
+| `prior_depth_path` | `str` | `None` | 深度图文件夹路径 |
+
+**渲染视频参数：**
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `save_rendered` | `bool` | `False` | 从高斯点云渲染插值飞行视频 |
+| `render_interp_per_pair` | `int` | `15` | 每对相机之间的插值帧数 |
+| `render_depth` | `bool` | `False` | 同时渲染深度可视化视频 |
+
+**其他参数：**
+
+| 参数 | 类型 | 默认值 | 描述 |
+|------|------|--------|------|
+| `log_time` | `bool` | `True` | 打印计时报告并保存 `pipeline_timing.json` |
+| `strict_output_path` | `str` | `None` | 若指定，结果直接保存到该路径下，不创建 `<case_name>/<timestamp>` 子目录 |
+
+---
+### 命令行参考
+所有 `__call__` 参数都可作为命令行参数使用：
+
+```bash
+python -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --output_path inference_output \
+    --target_size 952 \
+    --prior_cam_path path/to/camera_params.json \
+    --prior_depth_path path/to/depth_dir/
+```
+
+**布尔标志约定：**
+
+| 启用 | 禁用 |
+|------|------|
+| `--save_colmap` | *（省略）* |
+| `--save_conf` | *（省略）* |
+| `--save_sky_mask` | *（省略）* |
+| `--apply_sky_mask`（默认开启） | `--no_sky_mask` |
+| `--apply_edge_mask`（默认开启） | `--no_edge_mask` |
+| `--apply_confidence_mask` | *（省略）* |
+| `--compress_pts`（默认开启） | `--no_compress_pts` |
+| `--log_time`（默认开启） | `--no_log_time` |
+| *（默认开启）* `save_depth` | `--no_save_depth` |
+| *（默认开启）* `save_normal` | `--no_save_normal` |
+| *（默认开启）* `save_gs` | `--no_save_gs` |
+| *（默认开启）* `save_camera` | `--no_save_camera` |
+| *（默认开启）* `save_points` | `--no_save_points` |
+| `--save_rendered` | *（省略）* |
+| `--render_depth` | *（省略）* |
+
+**仅命令行参数：**
+
+| 参数 | 描述 |
+|------|------|
+| `--config_path` | 用于自定义检查点加载的训练配置 YAML |
+| `--ckpt_path` | 本地检查点文件路径 |
+| `--use_fsdp` | 启用 FSDP 多卡分片 |
+| `--enable_bf16` | 启用 bfloat16 混合精度 |
+| `--fsdp_cpu_offload` | 将 FSDP 参数卸载到 CPU |
+| `--disable_heads` | 以空格分隔要禁用的预测头（例如 `--disable_heads camera normal`） |
+| `--no_interactive` | 首次推理后退出（跳过交互式提示循环） |
+
+---
+### 输出格式
+#### 文件结构
+
+```
+inference_output/
+└── <case_name>/
+    └── <timestamp>/
+        ├── depth/
+        │   ├── depth_0000.png      # 归一化深度可视化
+        │   ├── depth_0000.npy      # 原始 float32 深度值 [H, W]
+        │   └── ...
+        ├── normal/
+        │   ├── normal_0000.png     # 法线图可视化（RGB）
+        │   └── ...
+        ├── camera_params.json      # 相机外参和内参
+        ├── gaussians.ply           # 3D 高斯点云（标准格式）
+        ├── points.ply              # 带颜色的点云
+        ├── sparse/                 # COLMAP 格式（使用 --save_colmap 时）
+        │   └── 0/
+        │       ├── cameras.bin
+        │       ├── images.bin
+        │       └── points3D.bin
+        ├── rendered/               # 渲染视频（使用 --save_rendered 时）
+        │   ���── rendered_rgb.mp4
+        │   └── rendered_depth.mp4  # （使用 --render_depth 时）
+        └── pipeline_timing.json    # 性能计时报告
+```
+
+#### 预测字典
+使用 Python API 时，`pipeline(...)` 内部生成一个 `predictions` 字典，包含以下键：
+
+```python
+# 几何
+predictions["depth"]        # [B, S, H, W, 1]  — 相机坐标系中的 Z 深度
+predictions["depth_conf"]   # [B, S, H, W]     — 深度置信度
+predictions["normals"]      # [B, S, H, W, 3]  — 相机坐标系中的表面法线
+predictions["normals_conf"] # [B, S, H, W]     — 法线置信度
+predictions["pts3d"]        # [B, S, H, W, 3]  — 世界坐标系中的 3D 点图
+predictions["pts3d_conf"]   # [B, S, H, W]     — 点云置信度
+# 相机
+predictions["camera_poses"] # [B, S, 4, 4]     — 相机到世界（c2w），OpenCV 约定
+predictions["camera_intrs"] # [B, S, 3, 3]     — 相机内参矩阵
+predictions["camera_params"]# [B, S, 9]        — 紧凑相机向量（平移、四元数、fov_v、fov_u）
+# 3D 高斯点云
+predictions["splats"]["means"]      # [B, N, 3] — 高斯中心
+predictions["splats"]["scales"]     # [B, N, 3] — 高斯尺度
+predictions["splats"]["quats"]      # [B, N, 4] — 高斯旋转（四元数）
+predictions["splats"]["opacities"]  # [B, N]    — 高斯不透明度
+predictions["splats"]["sh"]         # [B, N, 1, 3] — 球谐函数（0 阶）
+predictions["splats"]["weights"]    # [B, N]    — 逐高斯置信度权重
+```
+
+其中 `B` = 批大小（推理时始终为 1），`S` = 输入视图数，`H, W` = 图像尺寸，`N` = 总高斯数（`S × H × W`）。
+
+---
+### 先验注入
+WorldMirror 2.0 接受三种几何先验作为条件输入。先验会从提供的文件中自动检测。
+
+| 先验类型 | 条件标志 | 输入格式 |
+|----------|----------|----------|
+| 相机位姿 | `cond_flags[0]` | c2w 4×4 矩阵（OpenCV 约定） |
+| 深度图 | `cond_flags[1]` | 逐视图浮点深度图 |
+| 相机内参 | `cond_flags[2]` | 3×3 内参矩阵 |
+
+#### 相机参数（JSON）
+相机参数文件格式与 Pipeline 输出的 `camera_params.json` 一致：
+
+```json
+{
+  "num_cameras": 2,
+  "extrinsics": [
+    {
+      "camera_id": 0,
+      "matrix": [
+        [0.98, 0.01, -0.17, 0.52],
+        [-0.01, 0.99, 0.01, -0.03],
+        [0.17, -0.01, 0.98, 1.20],
+        [0.0, 0.0, 0.0, 1.0]
+      ]
+    }
+  ],
+  "intrinsics": [
+    {
+      "camera_id": 0,
+      "matrix": [
+        [525.0, 0.0, 320.0],
+        [0.0, 525.0, 240.0],
+        [0.0, 0.0, 1.0]
+      ]
+    }
+  ]
+}
+```
+
+**字段说明：**
+
+| 字段 | 描述 |
+|------|------|
+| `camera_id` | 整数索引（`0`、`1`、`2` ...）或图像文件名（不含扩展名，如 `"image_0001"`） |
+| `extrinsics.matrix` | 4×4 相机到世界（c2w）变换矩阵，OpenCV 坐标约定 |
+| `intrinsics.matrix` | 3×3 相机内参矩阵（像素单位）：`fx, fy` 为焦距，`cx, cy` 为主点坐标 |
+
+**重要说明：**
+- `extrinsics` 和 `intrinsics` 列表可以独立提供或一起提供。列表为空 `[]` 或缺失字段表示该先验不可用。
+- **内参分辨率：** 值应对应**原始图像分辨率**。Pipeline 会根据推理时的 resize + center-crop 自动调整。
+- **外参对齐：** Pipeline 会自动将所有外参相对于第一帧归一化，与训练行为一致。
+#### 深度图（文件夹）
+深度图以独立文件存储在一个文件夹中。文件名应与输入图像文件名对应。支持格式：`.npy`、`.exr`、`.png`（16-bit）。
+
+```
+prior_depth/
+├── image_0001.npy    # float32, shape [H, W]
+├── image_0002.npy
+└── ...
+```
+
+#### 组合先验
+先验可以自由组合。示例：
+
+```bash
+# 仅内参
+python -m hyworld2.worldrecon.pipeline --input_path images/ \
+    --prior_cam_path camera_intrinsics_only.json
+# 仅深度
+python -m hyworld2.worldrecon.pipeline --input_path images/ \
+    --prior_depth_path depth_maps/
+# 相机位姿 + 内参 + 深度
+python -m hyworld2.worldrecon.pipeline --input_path images/ \
+    --prior_cam_path camera_params.json \
+    --prior_depth_path depth_maps/
+```
+
+---
+### 多卡推理
+WorldMirror 2.0 支持跨多卡的**序列并行（SP）**推理，其中 token 序列在 ViT 骨干网络中跨 rank 分片，DPT 预测头并行处理帧。
+
+> **要求：** 输入图像数量必须 **>= GPU 数量**（`nproc_per_node`）。例如，使用 8 卡时需要提供至少 8 张输入图像。如果不满足此条件，Pipeline 将报错。
+
+```bash
+# 2 卡 + FSDP + bf16
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --use_fsdp --enable_bf16
+# 4 卡
+torchrun --nproc_per_node=4 -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --use_fsdp --enable_bf16
+# Python API（在 torchrun 脚本内）
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+pipeline = WorldMirrorPipeline.from_pretrained(
+    'tencent/HY-World-2.0',
+    use_fsdp=True,
+    enable_bf16=True,
+)
+pipeline('path/to/images')
+```
+
+**内部工作原理：**
+1. `from_pretrained` 自动检测 `WORLD_SIZE > 1` 并初始化 `torch.distributed`。
+2. 模型在 rank 0 上加载，并通过 `sync_module_states=True` 广播。
+3. FSDP 将参数跨 SP 进程组分片。
+4. DPT 预测头将帧分配到各 rank 并通过 `AllGather` 汇总结果。
+5. 后处理（掩码计算、保存）仅在 rank 0 上运行。
+
+---
+### 高级选项
+#### 禁用预测头
+当只需要特定输出时，可以禁用不需要的预测头以节省显存：
+
+```python
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+
+pipeline = WorldMirrorPipeline.from_pretrained(
+    'tencent/HY-World-2.0',
+    disable_heads=["normal", "points"],  # 释放约 200M 参数
+)
+```
+
+可禁用的预测头：`"camera"`、`"depth"`、`"normal"`、`"points"`、`"gs"`。
+#### 掩码过滤
+Pipeline 支持三种输出过滤方式，以提高点云和高斯质量：
+1. **天空掩码**（`apply_sky_mask=True`）：使用基于 ONNX 的分割模型移除天空区域，可选与模型预测的深度掩码融合。
+2. **边缘掩码**（`apply_edge_mask=True`）：移除深度/法线不连续处（物体边界）的点。
+3. **置信度掩码**（`apply_confidence_mask=False`）：移除预测置信度最低的 N% 的点。
+这些掩码独立应用于 `points.ply`（基于深度）和 `gaussians.ply`（基于 GS）输出。GS 输出在可用时使用其自身的深度预测进行边缘检测。
+#### 点云压缩
+当 `compress_pts=True`（默认）时，基于深度的点云会经过以下处理：
+1. **体素合并**：每个体素内的点（大小由 `compress_pts_voxel_size` 控制）通过加权平均进行合并。
+2. **随机下采样**：如果结果超过 `compress_pts_max_points`，则进行均匀下采样。
+类似地，高斯也会经过体素剪枝（均值、尺度、四元数、颜色、不透明度的加权平均），并可选下采样至 `compress_gs_max_points`。
+
+---
+### Gradio 应用
+WorldMirror 2.0 的交互式 Web 演示。上传图像或视频，即可在浏览器中可视化 3DGS、点云、深度图、法线图和相机参数。
+**快速开始：**
+
+```bash
+# 单卡
+python -m hyworld2.worldrecon.gradio_app
+
+# 多卡
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.gradio_app \
+    --use_fsdp --enable_bf16
+```
+
+**使用本地检查点：**
+
+```bash
+python -m hyworld2.worldrecon.gradio_app \
+    --config_path /path/to/config.yaml \
+    --ckpt_path /path/to/checkpoint.safetensors
+```
+
+**创建公开链接（如 Colab 或远程服务器）：**
+
+```bash
+python -m hyworld2.worldrecon.gradio_app --share
+```
+
+**参数：**
+
+| 参数 | 默认值 | 描述 |
+|------|--------|------|
+| `--port` | `8081` | 服务端口 |
+| `--host` | `0.0.0.0` | 服务主机 |
+| `--share` | `False` | 创建公开的 Gradio 链接 |
+| `--examples_dir` | `./examples/worldrecon` | 示例场景目录路径 |
+| `--config_path` | `None` | 训练配置 YAML（与 `--ckpt_path` 配合使用） |
+| `--ckpt_path` | `None` | 本地检查点文件（`.ckpt` / `.safetensors`） |
+| `--use_fsdp` | `False` | 启用 FSDP 多卡分片 |
+| `--enable_bf16` | `False` | 启用 bfloat16 混合精度 |
+| `--fsdp_cpu_offload` | `False` | 将 FSDP 参数卸载到 CPU（节省显存） |
+
+> **重要提示：** 在多卡模式下，输入图像数量必须 **>= GPU 数量**。
+
+---
+## 全景生成
+*即将发布。*
+本节将记录全景生成模型，包括：
+- 文本到全景和图像到全景的 API
+- 模型架构（基于 MMDiT 的隐式透视到 ERP 映射）
+- 配置参数
+- 输出格式
+
+---
+## 世界生成
+*即将发布。*
+本节将记录世界生成流水线，包括：
+- 轨迹规划配置
+- 基于记忆驱动的视频生成进行世界扩展
+- 世界组合（点云扩展 + 3DGS 优化）
+- 从文本/图像到可导航3D世界的端到端生成

HY-WorldMirror-2.0/README.md

@@ -0,0 +1,10 @@
+---
+tags:
+- model_hub_mixin
+- pytorch_model_hub_mixin
+---
+
+This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
+- Code: [More Information Needed]
+- Paper: [More Information Needed]
+- Docs: [More Information Needed]

HY-WorldMirror-2.0/config.json

@@ -0,0 +1,37 @@
+{
+  "condition_strategy": [
+    "token",
+    "pow3r",
+    "token"
+  ],
+  "depth": 24,
+  "disable_gs_depth": false,
+  "dpt_gradient_checkpoint": false,
+  "embed_dim": 1024,
+  "enable_bf16": false,
+  "enable_cam": true,
+  "enable_cond": true,
+  "enable_depth": true,
+  "enable_depth_mask": true,
+  "enable_gs": true,
+  "enable_norm": true,
+  "enable_pts": true,
+  "fixed_patch_embed": true,
+  "gs_dim": 256,
+  "img_size": 518,
+  "mlp_ratio": 4.0,
+  "model_size": "large",
+  "normalized_rope": true,
+  "num_heads": 16,
+  "num_register_tokens": 4,
+  "patch_embed": "dinov2_vitl14_reg",
+  "patch_size": 14,
+  "rope_base": 100.0,
+  "rope_jitter_coords": null,
+  "rope_normalize_coords": "separate",
+  "rope_rescale_coords": null,
+  "rope_shift_coords": null,
+  "sampling_strategy": "uniform",
+  "set_sky_region_to_maxdepth": false,
+  "sp_size": 1
+}

HY-WorldMirror-2.0/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fff06539d3d9e85338d7de1ffb5afffb7739fa5bf4d62b4b7c319b5ecdde54f
+size 5053553272

License.txt

@@ -0,0 +1,82 @@
+TENCENT HY-WORLD 2.0 COMMUNITY LICENSE AGREEMENT
+Tencent HY-WORLD 2.0 Release Date: April 15, 2026
+THIS LICENSE AGREEMENT DOES NOT APPLY IN THE EUROPEAN UNION, UNITED KINGDOM AND SOUTH KOREA AND IS EXPRESSLY LIMITED TO THE TERRITORY, AS DEFINED BELOW.
+By clicking to agree or by using, reproducing, modifying, distributing, performing or displaying any portion or element of the Tencent HY-WORLD 2.0 Works, including via any Hosted Service, You will be deemed to have recognized and accepted the content of this Agreement, which is effective immediately.
+1.	DEFINITIONS.
+a.	“Acceptable Use Policy” shall mean the policy made available by Tencent as set forth in the Exhibit A.
+b.	“Agreement” shall mean the terms and conditions for use, reproduction, distribution, modification, performance and displaying of Tencent HY-WORLD 2.0 Works or any portion or element thereof set forth herein.
+c.	“Documentation” shall mean the specifications, manuals and documentation for Tencent HY-WORLD 2.0 made publicly available by Tencent.
+d.	“Hosted Service” shall mean a hosted service offered via an application programming interface (API), web access, or any other electronic or remote means.
+e.	“Licensee,” “You” or “Your” shall mean a natural person or legal entity exercising the rights granted by this Agreement and/or using the Tencent HY-WORLD 2.0 Works for any purpose and in any field of use.
+f.	“Materials” shall mean, collectively, Tencent’s proprietary Tencent HY-WORLD 2.0 and Documentation (and any portion thereof) as made available by Tencent under this Agreement.
+g.	“Model Derivatives” shall mean all: (i) modifications to Tencent HY-WORLD 2.0 or any Model Derivative of Tencent HY-WORLD 2.0; (ii) works based on Tencent HY-WORLD 2.0 or any Model Derivative of Tencent HY-WORLD 2.0; or (iii) any other machine learning model which is created by transfer of patterns of the weights, parameters, operations, or Output of Tencent HY-WORLD 2.0 or any Model Derivative of Tencent HY-WORLD 2.0, to that model in order to cause that model to perform similarly to Tencent HY-WORLD 2.0 or a Model Derivative of Tencent HY-WORLD 2.0, including distillation methods, methods that use intermediate data representations, or methods based on the generation of synthetic data Outputs by Tencent HY-WORLD 2.0 or a Model Derivative of Tencent HY-WORLD 2.0 for training that model. For clarity, Outputs by themselves are not deemed Model Derivatives.
+h.	“Output” shall mean the information and/or content output of Tencent HY-WORLD 2.0 or a Model Derivative that results from operating or otherwise using Tencent HY-WORLD 2.0 or a Model Derivative, including via a Hosted Service.
+i.	“Tencent,” “We” or “Us” shall mean the applicable entity or entities in the Tencent corporate family that own(s) intellectual property or other rights embodied in or utilized by the Materials..
+j.	“Tencent HY-WORLD 2.0” shall mean the 3D generation models and their software and algorithms, including trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing made publicly available by Us at [https://github.com/Tencent-Hunyuan/HY-World-2.0].
+k.	“Tencent HY-WORLD 2.0 Works” shall mean: (i) the Materials; (ii) Model Derivatives; and (iii) all derivative works thereof.
+l.	“Territory” shall mean the worldwide territory, excluding the territory of the European Union, United Kingdom and South Korea. 
+m.	“Third Party” or “Third Parties” shall mean individuals or legal entities that are not under common control with Us or You.
+n.	“including” shall mean including but not limited to.
+2.	GRANT OF RIGHTS.
+We grant You, for the Territory only, a non-exclusive, non-transferable and royalty-free limited license under Tencent’s intellectual property or other rights owned by Us embodied in or utilized by the Materials to use, reproduce, distribute, create derivative works of (including Model Derivatives), and make modifications to the Materials, only in accordance with the terms of this Agreement and the Acceptable Use Policy, and You must not violate (or encourage or permit anyone else to violate) any term of this Agreement or the Acceptable Use Policy.
+3.	DISTRIBUTION.
+You may, subject to Your compliance with this Agreement, distribute or make available to Third Parties the Tencent HY-WORLD 2.0 Works, exclusively in the Territory, provided that You meet all of the following conditions:
+a.	You must provide all such Third Party recipients of the Tencent HY-WORLD 2.0 Works or products or services using them a copy of this Agreement;
+b.	You must cause any modified files to carry prominent notices stating that You changed the files;
+c.	You are encouraged to: (i) publish at least one technology introduction blogpost or one public statement expressing Your experience of using the Tencent HY-WORLD 2.0 Works; and (ii) mark the products or services developed by using the Tencent HY-WORLD 2.0 Works to indicate that the product/service is “Powered by Tencent HY”; and
+d.	All distributions to Third Parties (other than through a Hosted Service) must be accompanied by a “Notice” text file that contains the following notice: “Tencent HY-WORLD 2.0 is licensed under the Tencent HY-WORLD 2.0 Community License Agreement, Copyright © 2026 Tencent. All Rights Reserved. The trademark rights of “Tencent HY” are owned by Tencent or its affiliate.”
+e.	In the event that You use, integrate, implement, or otherwise deploy the Tencent HY Works, in whole or in part, to provide, enable, or support any service, product, or functionality to third parties, You shall clearly, accurately, and prominently disclose to all end users the full legal name and entity of the actual provider of such service, product, or functionality. You shall expressly and conspicuously state that Tencent is not affiliated with, associated with, sponsoring, or endorsing any such service, product, or functionality. You shall not use or display any name, logo, trademark, trade name, or other indicia of Tencent in any manner that could be construed as, or be likely to create, confusion, deception, or a false impression regarding any relationship, affiliation, sponsorship, or endorsement by Tencent.
+You may add Your own copyright statement to Your modifications and, except as set forth in this Section and in Section 5, may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Model Derivatives as a whole, provided Your use, reproduction, modification, distribution, performance and display of the work otherwise complies with the terms and conditions of this Agreement (including as regards the Territory). If You receive Tencent HY-WORLD 2.0 Works from a Licensee as part of an integrated end user product, then this Section 3 of this Agreement will not apply to You.
+4.	ADDITIONAL COMMERCIAL TERMS.
+If, on the Tencent HY-WORLD 2.0 version release date, the monthly active users of all products or services made available by or for Licensee is greater than 1 million monthly active users in the preceding calendar month, You must request a license from Tencent, which Tencent may grant to You in its sole discretion, and You are not authorized to exercise any of the rights under this Agreement unless or until Tencent otherwise expressly grants You such rights. 
+Subject to Tencent's written approval, you may request a license for the use of Tencent HY-WORLD 2.0 by submitting the following information to hunyuan3d@tencent.com:
+a.	Your company’s name and associated business sector that plans to use Tencent HY-WORLD 2.0.
+b.	Your intended use case and the purpose of using Tencent HY-WORLD 2.0.
+c.	Your plans to modify Tencent HY-WORLD 2.0 or create Model Derivatives.
+5.	RULES OF USE.
+a.	Your use of the Tencent HY-WORLD 2.0 Works must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Tencent HY-WORLD 2.0 Works, which is hereby incorporated by reference into this Agreement. You must include the use restrictions referenced in these Sections 5(a) and 5(b) as an enforceable provision in any agreement (e.g., license agreement, terms of use, etc.) governing the use and/or distribution of Tencent HY-WORLD 2.0 Works and You must provide notice to subsequent users to whom You distribute that Tencent HY-WORLD 2.0 Works are subject to the use restrictions in these Sections 5(a) and 5(b).
+b.	You must not use the Tencent HY-WORLD 2.0 Works or any Output or results of the Tencent HY-WORLD 2.0 Works to improve any other AI model (other than Tencent HY-WORLD 2.0 or Model Derivatives thereof).
+c.	You must not use, reproduce, modify, distribute, or display the Tencent HY-WORLD 2.0 Works, Output or results of the Tencent HY-WORLD 2.0 Works outside the Territory. Any such use outside the Territory is unlicensed and unauthorized under this Agreement.
+6.	INTELLECTUAL PROPERTY.
+a.	Subject to Tencent’s ownership of Tencent HY-WORLD 2.0 Works made by or for Tencent and intellectual property rights therein, conditioned upon Your compliance with the terms and conditions of this Agreement, as between You and Tencent, You will be the owner of any derivative works and modifications of the Materials and any Model Derivatives that are made by or for You.
+b.	No trademark licenses are granted under this Agreement, and in connection with the Tencent HY-WORLD 2.0 Works, Licensee may not use any name or mark owned by or associated with Tencent or any of its affiliates, except as required for reasonable and customary use in describing and distributing the Tencent HY-WORLD 2.0 Works. Tencent hereby grants You a license to use “Tencent HY” (the “Mark”) in the Territory solely as required to comply with the provisions of Section 3(c), provided that You comply with any applicable laws related to trademark protection. All goodwill arising out of Your use of the Mark will inure to the benefit of Tencent.
+c.	If You commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against Us or any person or entity alleging that the Materials or any Output, or any portion of any of the foregoing, infringe any intellectual property or other right owned or licensable by You, then all licenses granted to You under this Agreement shall terminate as of the date such lawsuit or other proceeding is filed. You will defend, indemnify and hold harmless Us from and against any claim by any Third Party arising out of or related to Your or the Third Party’s use or distribution of the Tencent HY-WORLD 2.0 Works.
+d.	Tencent claims no rights in Outputs You generate. You and Your users are solely responsible for Outputs and their subsequent uses.
+7.	DISCLAIMERS OF WARRANTY AND LIMITATIONS OF LIABILITY.
+a.	We are not obligated to support, update, provide training for, or develop any further version of the Tencent HY-WORLD 2.0 Works or to grant any license thereto.
+b.	UNLESS AND ONLY TO THE EXTENT REQUIRED BY APPLICABLE LAW, THE TENCENT HY-WORLD 2.0 WORKS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES OF ANY KIND INCLUDING ANY WARRANTIES OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, COURSE OF DEALING, USAGE OF TRADE, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING, REPRODUCING, MODIFYING, PERFORMING, DISPLAYING OR DISTRIBUTING ANY OF THE TENCENT HY-WORLD 2.0 WORKS OR OUTPUTS AND ASSUME ANY AND ALL RISKS ASSOCIATED WITH YOUR OR A THIRD PARTY’S USE OR DISTRIBUTION OF ANY OF THE TENCENT HY-WORLD 2.0 WORKS OR OUTPUTS AND YOUR EXERCISE OF RIGHTS AND PERMISSIONS UNDER THIS AGREEMENT.
+c.	TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL TENCENT OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, FOR ANY DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, CONSEQUENTIAL OR PUNITIVE DAMAGES, OR LOST PROFITS OF ANY KIND ARISING FROM THIS AGREEMENT OR RELATED TO ANY OF THE TENCENT HY-WORLD 2.0 WORKS OR OUTPUTS, EVEN IF TENCENT OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+8.	SURVIVAL AND TERMINATION.
+a.	The term of this Agreement shall commence upon Your acceptance of this Agreement or access to the Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein.
+b.	We may terminate this Agreement if You breach any of the terms or conditions of this Agreement. Upon termination of this Agreement, You must promptly delete and cease use of the Tencent HY-WORLD 2.0 Works. Sections 6(a), 6(c), 7 and 9 shall survive the termination of this Agreement.
+9.	GOVERNING LAW AND JURISDICTION.
+a.	This Agreement and any dispute arising out of or relating to it will be governed by the laws of the Hong Kong Special Administrative Region of the People’s Republic of China, without regard to conflict of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
+b.	Exclusive jurisdiction and venue for any dispute arising out of or relating to this Agreement will be a court of competent jurisdiction in the Hong Kong Special Administrative Region of the People’s Republic of China, and Tencent and Licensee consent to the exclusive jurisdiction of such court with respect to any such dispute.
+ 
+EXHIBIT A
+ACCEPTABLE USE POLICY
+
+Tencent reserves the right to update this Acceptable Use Policy from time to time.
+Last modified: December 30, 2025
+
+Tencent endeavors to promote safe and fair use of its tools and features, including Tencent HY-WORLD 2.0. You agree not to use Tencent HY-WORLD 2.0 or Model Derivatives:
+1.	Outside the Territory;
+2.	In any way that violates any applicable national, federal, state, local, international or any other law or regulation;
+3.	To harm Yourself or others;
+4.	To repurpose or distribute output from Tencent HY-WORLD 2.0 or any Model Derivatives to harm Yourself or others; 
+5.	To override or circumvent the safety guardrails and safeguards We have put in place;
+6.	For the purpose of exploiting, harming or attempting to exploit or harm minors in any way;
+7.	To generate or disseminate verifiably false information and/or content with the purpose of harming others or influencing elections;
+8.	To generate or facilitate false online engagement, including fake reviews and other means of fake online engagement;
+9.	To intentionally defame, disparage or otherwise harass others;
+10.	To generate and/or disseminate malware (including ransomware) or any other content to be used for the purpose of harming electronic systems;
+11.	To generate or disseminate personal identifiable information with the purpose of harming others;
+12.	To generate or disseminate information (including images, code, posts, articles), and place the information in any public context (including –through the use of bot generated tweets), without expressly and conspicuously identifying that the information and/or content is machine generated;
+13.	To impersonate another individual without consent, authorization, or legal right;
+14.	To make high-stakes automated decisions in domains that affect an individual’s safety, rights or wellbeing (e.g., law enforcement, migration, medicine/health, management of critical infrastructure, safety components of products, essential services, credit, employment, housing, education, social scoring, or insurance);
+15.	In a manner that violates or disrespects the social ethics and moral standards of other countries or regions;
+16.	To perform, facilitate, threaten, incite, plan, promote or encourage violent extremism or terrorism;
+17.	For any use intended to discriminate against or harm individuals or groups based on protected characteristics or categories, online or offline social behavior or known or predicted personal or personality characteristics;
+18.	To intentionally exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
+19.	For military purposes;
+20.	To engage in the unauthorized or unlicensed practice of any profession including, but not limited to, financial, legal, medical/health, or other professional practices.

README.md

@@ -0,0 +1,490 @@
+---
+language:
+- en
+- zh
+license: other
+license_name: tencent-hy-world-2.0-community
+license_link: https://github.com/Tencent-Hunyuan/HY-World-2.0/blob/main/License.txt
+pipeline_tag: image-to-3d
+library_name: hy-world-2
+tags:
+- worldmodel
+- 3d
+- hy-world
+extra_gated_eu_disallowed: true
+---
+
+<h1>HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds</h1>
+
+[English](README.md) | [简体中文](README_zh.md)
+
+<p align="center">
+  <img src="assets/teaser.png" width="95%" alt="HY-World-2.0 Teaser">
+</p>
+
+<div align="center">
+  <a href=https://3d.hunyuan.tencent.com/sceneTo3D target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a>
+  <a href=https://huggingface.co/tencent/HY-World-2.0 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
+  <a href=https://3d-models.hunyuan.tencent.com/world/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
+  <a href=https://3d-models.hunyuan.tencent.com/world/world2_0/HY_World_2_0.pdf target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
+   <a href=https://modelscope.cn/models/Tencent-Hunyuan/HY-World-2.0 target="_blank"><img src=https://img.shields.io/badge/ModelScope-Models-624aff.svg height=22px></a>
+  <a href=https://discord.gg/dNBrdrGGMa target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
+  <a href=https://x.com/TencentHunyuan target="_blank"><img src=https://img.shields.io/badge/Tencent%20HY-black.svg?logo=x height=22px></a>
+ <a href="#community-resources" target="_blank"><img src=https://img.shields.io/badge/Community-lavender.svg?logo=homeassistantcommunitystore height=22px></a>
+</div>
+
+<br>
+<p align="center">
+  <i>"What Is Now Proved Was Once Only Imagined"</i>
+</p>
+
+## 🎥 Video
+<video width="100%" controls><source src="https://github.com/user-attachments/assets/b56f4750-25c9-48fb-83ff-d58526711463" type="video/mp4"></video>
+
+## 🔥 News
+
+- **[April 15, 2026]**: 🚀 Release HY-World 2.0 technical report & partial codes!
+- **[April 15, 2026]**: 🤗 Open-source WorldMirror 2.0 inference code and model weights!
+- **[Coming Soon]**: Release Full HY-World 2.0 (World Generation) inference code.
+- **[Coming Soon]**: Release ![Panorama Generation](https://img.shields.io/badge/Panorama_Generation-4285F4?style=flat-square) (HY-Pano 2.0) model weights & code.
+- **[Coming Soon]**: Release ![Trajectory Planning](https://img.shields.io/badge/Trajectory_Planning-EA4335?style=flat-square)（WorldNav） code.
+- **[Coming Soon]**: Release ![World Expansion](https://img.shields.io/badge/World_Expansion-FBBC05?style=flat-square)(WorldStereo 2.0) model weights & inference code.
+
+
+## 📋 Table of Contents
+- [📖 Introduction](#-introduction)
+- [✨ Highlights](#-highlights)
+- [🧩 Architecture](#-architecture)
+- [📝 Open-Source Plan](#-open-source-plan)
+- [🎁 Model Zoo](#-model-zoo)
+- [🤗 Get Started](#-get-started)
+- [🔮 Performance](#-performance)
+- [🎬 More Examples](#-more-examples)
+- [📚 Citation](#-citation)
+
+
+## 📖 Introduction
+
+**HY-World 2.0** is a multi-modal world model framework for **world generation** and **world reconstruction**. It accepts diverse input modalities — text, single-view images, multi-view images, and videos — and produces 3D world representations (meshes / Gaussian Splattings). It offers two core capabilities:
+
+- **World Generation** (text / single image &rarr; 3D world): syntheses high-fidelity, navigable 3D scenes through a four-stage method —— a) ![Panorama Generation](https://img.shields.io/badge/Panorama_Generation-4285F4?style=flat-square) with HY-Pano 2.0, b) ![Trajectory Planning](https://img.shields.io/badge/Trajectory_Planning-EA4335?style=flat-square) with WorldNav, c) ![World Expansion](https://img.shields.io/badge/World_Expansion-FBBC05?style=flat-square) with WorldStereo 2.0, and d) ![World Composition](https://img.shields.io/badge/World_Composition-34A853?style=flat-square) with WorldMirror 2.0 & 3DGS learning.
+- **World Reconstruction** (multi-view images / video &rarr; 3D): Powered by WorldMirror 2.0, a unified feed-forward model that simultaneously predicts depth, surface normals, camera parameters, 3D point clouds, and 3DGS attributes in a single forward pass.
+
+HY-World 2.0 is the **first open-source state-of-the-art** 3D world model, delivering results comparable to closed-source methods such as Marble.  We will release all model weights, code, and technical details to facilitate reproducibility and advance research in this field.
+
+### Why 3D World Models?
+
+Existing world models, such as Genie 3, Cosmos, and HY-World 1.5 (WorldPlay+WorldCompass), generate pixel-level videos — essentially "watching a movie" that vanishes once playback ends. **HY-World 2.0 takes a fundamentally different approach**: it directly produces editable, persistent 3D assets (meshes / 3DGS) that can be imported into game engines like Blender/Unity/Unreal Engine/Isaac Sim — more like "building a playable game" than recording a clip. This paradigm shift natively resolves many long-standing pain points of video world models:
+
+|  | Video World Models | 3D World Model (HY-World 2.0) |
+|--|---|---|
+| **Output** | Pixel videos (non-editable) | Real 3D assets — meshes / 3DGS (fully editable) |
+| **Playable Duration** | Limited (typically < 1 min) | Unlimited — assets persist permanently |
+| **3D Consistency** | Poor (flickering, artifacts across views) | Native — inherently consistent in 3D |
+| **Real-Time Rendering** | Requires per-frame inference; high latency | Consumer GPUs can render in real time |
+| **Controllability** | Weak (imprecise character control, no real physics) | Precise — zero-error control, real physics collision, accurate lighting |
+| **Inference Cost** | Accumulates with every interaction | One-time generation; rendering cost ≈ 0 |
+| **Engine Compatibility** | ✗ Video files only | ✓ Directly importable into Blender / UE / Isaac Engine |
+| | $\color{IndianRed}{\textsf{Watch a video, then it's gone}}$ | $\color{RoyalBlue}{\textbf{Build a world, keep it forever}}$ |
+
+
+<table align="center" style="border: none;">
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_1.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_2.gif" width="100%"></td>
+  </tr>
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_7.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_8.gif" width="100%"></td>
+  </tr>
+</table>
+
+<p align="center"><em>All above are <strong>real 3D assets</strong> (not generated videos) and entirely created by HY-World 2.0 -- captured from live real-time interaction.</em></p>
+
+## ✨ Highlights
+
+- **Real 3D Worlds, Not Just Videos**
+
+  Unlike video-only world models (e.g., Genie 3, HY World 1.5), HY-World 2.0 generates **real 3D assets** — 3DGS, meshes, and point clouds — that are freely explorable, editable, and directly importable into **Unity / Unreal Engine / Isaac**. From a single text prompt or image, create navigable 3D worlds with diverse styles: realistic, cartoon, game, and more.
+
+<p align="center">
+  <img src="assets/mesh_en.gif" width="95%">
+</p>
+
+
+- **Instant 3D Reconstruction from Photos & Videos**
+
+  Powered by **WorldMirror 2.0**, a unified feed-forward model that predicts dense point clouds, depth maps, surface normals, camera parameters, and 3DGS from multi-view images or casual videos in a single forward pass. Supports flexible-resolution inference (50K–500K pixels) with SOTA accuracy. Capture a video, get a digital twin.
+
+<p align="center">
+  <img src="assets/recon_en.gif" width="95%">
+</p>
+
+- **Interactive Character Exploration**
+
+  Go beyond viewing — **play inside your generated worlds**. HY-World 2.0 supports first-person navigation and third-person character mode, enabling users to freely explore AI-generated streets, buildings, and landscapes with physics-based collision.  Go to [our product page]() for free try.
+
+<p align="center">
+  <img src="assets/interactive.gif" width="95%">
+</p>
+
+
+## 🧩 Architecture
+- **Refer to our tech report for more details**
+
+  A systematic pipeline of HY-World 2.0 — *Panorama Generation* (HY-Pano-2.0) &rarr; *Trajectory Planning* (WorldNav) &rarr; *World Expansion* (WorldStereo 2.0) &rarr; *World Composition* (WorldMirror 2.0 + 3DGS) — that automatically transforms text or a single image into a high-fidelity, navigable 3D world (3DGS/mesh outputs).
+
+<p align="center">
+  <img src="assets/overview.png" width="95%">
+</p>
+
+## 📝 Open-Source Plan
+
+- ✅ Technical Report
+- ✅ WorldMirror 2.0 Code & Model Checkpoints
+- ⬜ Full Inference Code for World Generation (WorldNav + World Composition)
+- ⬜ Panorama Generation (HY-Pano 2.0) Model & Code — [HunyuanWorld 1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0) available as interim alternative
+- ⬜ World Expansion (WorldStereo 2.0) Model & Code — [WorldStereo](https://github.com/FuchengSu/WorldStereo) available as interim alternative
+
+
+## 🎁 Model Zoo
+
+### World Reconstruction — WorldMirror Series
+
+| Model | Description | Params | Date | Hugging Face |
+|-------|-------------|--------|------|--------------|
+| WorldMirror 2.0 | Multi-view / video &rarr; 3D reconstruction | ~1.2B | 2026 | [Download](https://huggingface.co/tencent/HY-World-2.0/tree/main/HY-WorldMirror-2.0) |
+| WorldMirror 1.0 | Multi-view / video &rarr; 3D reconstruction (legacy) | ~1.2B | 2025 | [Download](https://huggingface.co/tencent/HunyuanWorld-Mirror/tree/main) |
+
+### Panorama Generation
+
+| Model | Description | Params | Date | Hugging Face |
+|-------|-------------|--------|------|--------------|
+| HY-PanoGen | Text / image &rarr; 360° panorama | — | Coming Soon | — |
+
+### World Generation
+
+| Model           | Description | Params | Date | Hugging Face |
+|-----------------|-------------|-----|------|--------------|
+| WorldStereo 2.0 | Panorama &rarr; navigable 3DGS world |  —  | Coming Soon | — |
+
+We recommend referring to our previous works, [WorldStereo](https://github.com/FuchengSu/WorldStereo) and [WorldMirror](https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror), for background knowledge on world generation and reconstruction. 
+
+## 🤗 Get Started
+
+### Install Requirements
+
+We recommend CUDA 12.4 for installation.
+
+```bash
+# 1. Clone the repository
+git clone https://github.com/Tencent-Hunyuan/HY-World-2.0
+cd HY-World-2.0
+
+# 2. Create conda environment
+conda create -n hyworld2 python=3.10
+conda activate hyworld2
+
+# 3. Install PyTorch (CUDA 12.4)
+pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
+
+# 4. Install dependencies
+pip install -r requirements.txt
+
+# 5. Install FlashAttention
+# (Recommended) Install FlashAttention-3
+git clone https://github.com/Dao-AILab/flash-attention.git
+cd flash-attention/hopper
+python setup.py install
+cd ../../
+rm -rf flash-attention
+
+# For simpler installation, you can also use FlashAttention-2
+pip install flash-attn --no-build-isolation
+```
+
+### Code Usage — Panorama Generation (HY-Pano-2)
+
+*Coming soon.*
+
+### Code Usage — World Generation (WorldNav, WorldStereo-2, and 3DGS)
+
+*Coming soon.*
+
+**We recommend referring to our previous work, [WorldStereo](https://github.com/FuchengSu/WorldStereo), for the open-source preview version of WorldStereo-2.**
+
+### Code Usage — WorldMirror 2.0
+WorldMirror 2.0 supports the following usage modes:
+
+- [Code Usage](#code-usage--worldmirror-20)
+- [Gradio App](#gradio-app--worldmirror-20)
+
+We provide a `diffusers`-like Python API for WorldMirror 2.0. Model weights are automatically downloaded from Hugging Face on first run.
+
+```python
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+
+pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
+result = pipeline('path/to/images')
+```
+
+**With Prior Injection (Camera & Depth):**
+
+```python
+result = pipeline(
+    'path/to/images',
+    prior_cam_path='path/to/prior_camera.json',
+    prior_depth_path='path/to/prior_depth/',
+)
+```
+
+> For the detailed structure of camera/depth priors and how to prepare them, see [Prior Preparation Guide](DOCUMENTATION.md#prior-injection).
+
+**CLI:**
+
+```bash
+# Single GPU
+python -m hyworld2.worldrecon.pipeline --input_path path/to/images
+
+# Multi-GPU
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --use_fsdp --enable_bf16
+```
+
+> **Important:** In multi-GPU mode, the number of input images must be **>= the number of GPUs**. For example, with `--nproc_per_node=8`, provide at least 8 images.
+
+### Gradio App — WorldMirror 2.0
+
+We provide an interactive [Gradio](https://www.gradio.app/) web demo for WorldMirror 2.0. Upload images or videos and visualize 3DGS, point clouds, depth maps, normal maps, and camera parameters in your browser.
+
+```bash
+# Single GPU
+python -m hyworld2.worldrecon.gradio_app
+
+# Multi-GPU
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.gradio_app \
+    --use_fsdp --enable_bf16
+```
+
+For the full list of Gradio app arguments (port, share, local checkpoints, etc.), see [DOCUMENTATION.md](DOCUMENTATION.md#gradio-app).
+
+
+
+## 🔮 Performance
+
+For full benchmark results, please refer to the [technical report](https://3d-models.hunyuan.tencent.com/world/).
+
+### WorldStereo 2.0 — Camera Control
+
+<table>
+  <thead>
+    <tr>
+      <th rowspan="2">Methods</th>
+      <th colspan="3" align="center">Camera Metrics</th>
+      <th colspan="4" align="center">Visual Quality</th>
+    </tr>
+    <tr>
+      <th>RotErr ↓</th><th>TransErr ↓</th><th>ATE ↓</th>
+      <th>Q-Align ↑</th><th>CLIP-IQA+ ↑</th><th>Laion-Aes ↑</th><th>CLIP-I ↑</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr><td>SEVA</td><td>1.690</td><td>1.578</td><td>2.879</td><td>3.232</td><td>0.479</td><td>4.623</td><td>77.16</td></tr>
+    <tr><td>Gen3C</td><td>0.944</td><td>1.580</td><td>2.789</td><td>3.353</td><td>0.489</td><td>4.863</td><td>82.33</td></tr>
+    <tr><td>WorldStereo</td><td>0.762</td><td>1.245</td><td>2.141</td><td>4.149</td><td><b>0.547</b></td><td>5.257</td><td>89.05</td></tr>
+    <tr><td><b>WorldStereo 2.0</b></td><td><b>0.492</b></td><td><b>0.968</b></td><td><b>1.768</b></td><td><b>4.205</b></td><td>0.544</td><td><b>5.266</b></td><td><b>89.43</b></td></tr>
+  </tbody>
+</table>
+
+### WorldStereo 2.0 — Single-View-Generated Reconstruction
+
+<table>
+  <thead>
+    <tr>
+      <th rowspan="2">Methods</th>
+      <th colspan="4">Tanks-and-Temples</th>
+      <th colspan="4">MipNeRF360</th>
+    </tr>
+    <tr>
+      <th>Precision ↑</th>
+      <th>Recall ↑</th>
+      <th>F1-Score ↑</th>
+      <th>AUC ↑</th>
+      <th>Precision ↑</th>
+      <th>Recall ↑</th>
+      <th>F1-Score ↑</th>
+      <th>AUC ↑</th>
+    </tr>
+  </thead>
+  <tbody align="center">
+    <tr>
+      <td align="left">SEVA</td>
+      <td>33.59</td>
+      <td>35.34</td>
+      <td>36.73</td>
+      <td>51.03</td>
+      <td>22.38</td>
+      <td>55.63</td>
+      <td>28.75</td>
+      <td>46.81</td>
+    </tr>
+    <tr>
+      <td align="left">Gen3C</td>
+      <td><u>46.73</u></td>
+      <td>25.51</td>
+      <td>31.24</td>
+      <td>42.44</td>
+      <td>23.28</td>
+      <td><strong>75.37</strong></td>
+      <td>35.26</td>
+      <td>52.10</td>
+    </tr>
+    <tr>
+      <td align="left">Lyra</td>
+      <td><strong>50.38</strong></td>
+      <td>28.67</td>
+      <td>32.54</td>
+      <td>43.05</td>
+      <td>30.02</td>
+      <td>58.60</td>
+      <td>36.05</td>
+      <td>49.89</td>
+    </tr>
+    <tr>
+      <td align="left">FlashWorld</td>
+      <td>26.58</td>
+      <td>20.72</td>
+      <td>22.29</td>
+      <td>30.45</td>
+      <td>35.97</td>
+      <td>53.77</td>
+      <td>42.60</td>
+      <td>53.86</td>
+    </tr>
+    <tr>
+      <td align="left">WorldStereo 2.0</td>
+      <td>43.62</td>
+      <td><u>41.02</u></td>
+      <td><u>41.43</u></td>
+      <td><u>58.19</u></td>
+      <td><strong>43.19</strong></td>
+      <td><u>65.32</u></td>
+      <td><strong>51.27</strong></td>
+      <td><strong>65.79</strong></td>
+    </tr>
+    <tr>
+      <td align="left">WorldStereo 2.0 (DMD)</td>
+      <td>40.41</td>
+      <td><strong>44.41</strong></td>
+      <td><strong>43.16</strong></td>
+      <td><strong>60.09</strong></td>
+      <td><u>42.34</u></td>
+      <td>64.83</td>
+      <td><u>50.52</u></td>
+      <td><u>65.64</u></td>
+    </tr>
+  </tbody>
+</table>
+
+### WorldMirror 2.0 — Point Map Reconstruction
+
+**Point Map Reconstruction on 7-Scenes, NRGBD, and DTU.** We report the mean Accuracy and Completeness of WorldMirror under different input configurations. **Bold** results are best. "L / M / H" denote low / medium / high inference resolution. "+ all priors" denotes injection of camera extrinsics, camera intrinsics, and depth priors.
+
+<table>
+  <thead>
+    <tr>
+      <th rowspan="2">Method</th>
+      <th colspan="2" align="center">7-Scenes <sub>(scene)</sub></th>
+      <th colspan="2" align="center">NRGBD <sub>(scene)</sub></th>
+      <th colspan="2" align="center">DTU <sub>(object)</sub></th>
+    </tr>
+    <tr>
+      <th>Acc. ↓</th><th>Comp. ↓</th>
+      <th>Acc. ↓</th><th>Comp. ↓</th>
+      <th>Acc. ↓</th><th>Comp. ↓</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr><td colspan="7"><em>WorldMirror 1.0</em></td></tr>
+    <tr><td>&nbsp;&nbsp;L</td><td>0.043</td><td>0.055</td><td>0.046</td><td>0.049</td><td>1.476</td><td>1.768</td></tr>
+    <tr><td>&nbsp;&nbsp;L + all priors</td><td>0.021</td><td>0.026</td><td>0.022</td><td>0.020</td><td>1.347</td><td>1.392</td></tr>
+    <tr><td>&nbsp;&nbsp;M</td><td>0.043</td><td>0.049</td><td>0.041</td><td>0.045</td><td>1.017</td><td>1.780</td></tr>
+    <tr><td>&nbsp;&nbsp;M + all priors</td><td>0.018</td><td>0.023</td><td>0.016</td><td>0.014</td><td>0.735</td><td>0.935</td></tr>
+    <tr><td>&nbsp;&nbsp;H</td><td>0.079</td><td>0.087</td><td>0.077</td><td>0.093</td><td>2.271</td><td>2.113</td></tr>
+    <tr><td>&nbsp;&nbsp;H + all priors</td><td>0.042</td><td>0.041</td><td>0.078</td><td>0.082</td><td>1.773</td><td>1.478</td></tr>
+    <tr><td colspan="7"></td></tr>
+    <tr><td colspan="7"><em>WorldMirror 2.0</em></td></tr>
+    <tr><td>&nbsp;&nbsp;L</td><td>0.041</td><td>0.052</td><td>0.047</td><td>0.058</td><td>1.352</td><td>2.009</td></tr>
+    <tr><td>&nbsp;&nbsp;L + all priors</td><td>0.019</td><td>0.024</td><td>0.017</td><td>0.015</td><td>1.100</td><td>1.201</td></tr>
+    <tr><td>&nbsp;&nbsp;M</td><td>0.033</td><td>0.046</td><td>0.039</td><td>0.047</td><td>1.005</td><td>1.892</td></tr>
+    <tr><td>&nbsp;&nbsp;M + all priors</td><td>0.013</td><td>0.017</td><td><b>0.013</b></td><td><b>0.013</b></td><td>0.690</td><td>0.876</td></tr>
+    <tr><td>&nbsp;&nbsp;H</td><td>0.037</td><td>0.040</td><td>0.046</td><td>0.053</td><td>0.845</td><td>1.904</td></tr>
+    <tr><td>&nbsp;&nbsp;<b>H + all priors</b></td><td><b>0.012</b></td><td><b>0.016</b></td><td>0.015</td><td>0.016</td><td><b>0.554</b></td><td><b>0.771</b></td></tr>
+  </tbody>
+</table>
+ 
+### WorldMirror 2.0 — Prior Comparison
+
+**Comparison with Pow3R and MapAnything under Different Prior Conditions.** Results are averaged on 7-Scenes, NRGBD, and DTU datasets. Pow3R (pro) refers to the original Pow3R with Procrustes alignment.
+
+
+<p align="center">
+  <img src="assets/prior_comparison2_wm2.png" width="85%">
+</p>
+
+
+
+
+## 🎬 More Examples
+
+<table align="center" style="border: none;">
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_3.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_4.gif" width="100%"></td>
+  </tr>
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_5.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_6.gif" width="100%"></td>
+  </tr>
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_9.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_10.gif" width="100%"></td>
+  </tr>
+</table>
+
+
+## 📖 Documentation
+
+For detailed usage guides, parameter references, output format specifications, and prior injection instructions, see **[DOCUMENTATION.md](DOCUMENTATION.md)**.
+
+
+## 📚 Citation
+
+If you find HunyuanWorld 2.0 useful for your research, please cite:
+
+```bibtex
+@article{hyworld22026,
+  title={HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds},
+  author={Tencent HY-World Team},
+  journal={arXiv preprint},
+  year={2026}
+}
+
+@article{hunyuanworld2025tencent,
+    title={HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels},
+    author={Team HunyuanWorld},
+    year={2025},
+    journal={arXiv preprint}
+}
+```
+
+
+## 📧 Contact
+
+Please send emails to tengfeiwang12@gmail.com for questions or feedback.
+
+
+## 🙏 Acknowledgements
+
+We would like to thank [HunyuanWorld 1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0), [WorldMirror](https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror), [WorldPlay](https://github.com/Tencent-Hunyuan/HY-WorldPlay), [WorldStereo](https://github.com/FuchengSu/WorldStereo), [HunyuanImage](https://github.com/Tencent-Hunyuan/HunyuanImage-3.0) for their great work.

README_zh.md

@@ -0,0 +1,475 @@
+
+
+<h1>HY-World 2.0：用于重建、生成和模拟3D世界的多模态世界模型</h1>
+
+[English](README.md) | [简体中文](README_zh.md)
+
+<p align="center">
+  <img src="assets/teaser.png" width="95%" alt="HY-World-2.0 Teaser">
+</p>
+
+<div align="center">
+  <a href=https://3d.hunyuan.tencent.com/sceneTo3D target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a>
+  <a href=https://huggingface.co/tencent/HY-World-2.0 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
+  <a href=https://3d-models.hunyuan.tencent.com/world/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
+  <a href=https://3d-models.hunyuan.tencent.com/world/world2_0/HY_World_2_0.pdf target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
+  <a href=https://discord.gg/dNBrdrGGMa target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
+  <a href=https://x.com/TencentHunyuan target="_blank"><img src=https://img.shields.io/badge/Tencent%20HY-black.svg?logo=x height=22px></a>
+ <a href="#community-resources" target="_blank"><img src=https://img.shields.io/badge/Community-lavender.svg?logo=homeassistantcommunitystore height=22px></a>
+</div>
+
+<br>
+<p align="center">
+  <i>"What Is Now Proved Was Once Only Imagined"</i>
+</p>
+
+## 🎥 视频
+<video width="100%" controls><source src="https://github.com/user-attachments/assets/b56f4750-25c9-48fb-83ff-d58526711463" type="video/mp4"></video>
+
+## 🔥 最新动态
+
+- **[2026年4月15日]**：🚀 发布 HY-World 2.0 技术报告及部分代码！
+- **[2026年4月15日]**：🤗 开源 WorldMirror 2.0 推理代码和模型权重！
+- **[即将发布]**：发布完整的 HY-World 2.0（World Generation）推理代码。
+- **[即将发布]**：发布 ![全景生成](https://img.shields.io/badge/Panorama_Generation-4285F4?style=flat-square)（HY-Pano 2.0）模型权重和代码。
+- **[即将发布]**：发布 ![轨迹规划](https://img.shields.io/badge/Trajectory_Planning-EA4335?style=flat-square)（WorldNav）代码。
+- **[即将发布]**：发布 ![世界扩展](https://img.shields.io/badge/World_Expansion-FBBC05?style=flat-square)（WorldStereo 2.0）模型权重和推理代码。
+
+
+## 📋 目录
+- [📖 介绍](#-介绍)
+- [✨ 亮点](#-亮点)
+- [🧩 架构](#-架构)
+- [📝 开源计划](#-开源计划)
+- [🎁 模型库](#-模型库)
+- [🤗 快速开始](#-快速开始)
+- [🔮 性能表现](#-性能表现)
+- [🎬 更多示例](#-更多示例)
+- [📚 引用](#-引用)
+
+
+## 📖 介绍
+
+**HY-World 2.0** 是一个面向**世界生成**和**世界重建**的多模态世界模型框架。它接受多种输入模态——文本、单视图图像、多视图图像和视频——并生成3D世界表示（网格 / 3D高斯点云）。它提供两大核心能力：
+
+- **世界生成**（文本 / 单张图像 &rarr; 3D 世界）：通过四阶段方法合成高保真、可导航的3D场景——a) ![全景生成](https://img.shields.io/badge/Panorama_Generation-4285F4?style=flat-square)（HY-Pano 2.0），b) ![轨迹规划](https://img.shields.io/badge/Trajectory_Planning-EA4335?style=flat-square)（WorldNav），c) ![世界扩展](https://img.shields.io/badge/World_Expansion-FBBC05?style=flat-square)（WorldStereo 2.0），d) ![世界组合](https://img.shields.io/badge/World_Composition-34A853?style=flat-square)（WorldMirror 2.0 + 3DGS 学习）。
+- **世界重建**（多视图图像 / 视频 &rarr; 3D）：由 WorldMirror 2.0 驱动，这是一个统一的前馈模型，能够在单次前向传播中同时预测深度、表面法线、相机参数、3D点云和3DGS属性。
+
+HY-World 2.0 是**首个开源的最先进**3D世界模型，其效果可与 Marble 等闭源方法相媲美。我们将发布所有模型权重、代码和技术细节，以促进可复现性和推动该领域的研究进展。
+
+### 为什么需要3D世界模型？
+
+现有的世界模型（如 Genie 3、Cosmos、HY-World 1.5（WorldPlay+WorldCompass））生成的是像素级视频——本质上是"看一部电影"，播放结束即消失。**HY-World 2.0 采用了完全不同的方法**：它直接生成可编辑、可持久化的3D资产（网格 / 3DGS），可以直接导入到 Blender/Unity/Unreal Engine/Isaac Sim 等游戏引擎中——更像是"构建一个可玩的游戏"，而非录制一段视频。这种范式转变从根本上解决了视频世界模型的许多长期痛点：
+
+|  | 视频世界模型 | 3D 世界模型（HY-World 2.0） |
+|--|---|---|
+| **输出** | 像素视频（不可编辑） | 真实 3D 资产——网格 / 3DGS（完全可编辑） |
+| **可交互时长** | 有限（通常 < 1 分钟） | 无限——资产永久保存 |
+| **3D 一致性** | 差（闪烁、跨视角伪影） | 原生一致——内在3D一致性 |
+| **实时渲染** | 需要逐帧推理；延迟高 | 消费级 GPU 即可实时渲染 |
+| **可控性** | 弱（角色控制不精确，无真实物理） | 精确——零误差控制、真实物理碰撞、准确光照 |
+| **推理成本** | 随每次交互累积 | 一次生成；渲染成本 ≈ 0 |
+| **引擎兼容性** | ✗ 仅视频文件 | ✓ 可直接导入 Blender / UE / Isaac Engine |
+| | $\color{IndianRed}{\textsf{看完视频，即刻消失}}$ | $\color{RoyalBlue}{\textbf{构建世界，永久保留}}$ |
+
+
+<table align="center" style="border: none;">
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_1.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_2.gif" width="100%"></td>
+  </tr>
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_7.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_8.gif" width="100%"></td>
+  </tr>
+</table>
+
+<p align="center"><em>以上均为<strong>真实3D资产</strong>（非生成视频），完全由 HY-World 2.0 创建——截取自实时交互画面。</em></p>
+
+## ✨ 亮点
+
+- **真实3D世界，而非仅仅是视频**
+
+  与纯视频世界模型（如 Genie 3、HY World 1.5）不同，HY-World 2.0 生成**真实3D资产**——3DGS、网格和点云——可自由浏览、编辑，并直接导入 **Unity / Unreal Engine / Isaac**。从一段文本提示或一张图像出发，即可创建多种风格的可导航3D世界：写实、卡通、游戏等。
+
+<p align="center">
+  <img src="assets/mesh.gif" width="95%">
+</p>
+
+
+- **从照片和视频即时3D重建**
+
+  由 **WorldMirror 2.0** 驱动，这是一个统一的前馈模型，能够在单次前向传播中从多视图图像或随手拍摄的视频中预测稠密点云、深度图、表面法线、相机参数和3DGS。支持灵活分辨率推理（50K–500K 像素），精度达到 SOTA 水平。拍摄一段视频，即可获得数字孪生。
+
+<p align="center">
+  <img src="assets/recon.gif" width="95%">
+</p>
+
+- **交互式角色探索**
+
+  不仅仅是观看——**在生成的世界中自由漫游**。HY-World 2.0 支持第一人称导航和第三人称角色模式，用户可以在 AI 生成的街道、建筑和景观中自由探索，并具备基于物理的碰撞效果。前往[我们的产品页面]()免费体验。
+
+<p align="center">
+  <img src="assets/interactive.gif" width="95%">
+</p>
+
+## 🧩 架构
+- **详细信息请参阅我们的技术报告**
+
+  HY-World 2.0 的系统化流水线——*全景生成*（HY-Pano-2.0）&rarr; *轨迹规划*（WorldNav）&rarr; *世界扩展*（WorldStereo 2.0）&rarr; *世界组合*（WorldMirror 2.0 + 3DGS）——能够自动将文本或单张图像转化为高保真、可导航的3D世界（3DGS/网格输出）。
+
+<p align="center">
+  <img src="assets/overview.png" width="95%">
+</p>
+
+## 📝 开源计划
+
+- ✅ 技术报告
+- ✅ WorldMirror 2.0 代码和模型权重
+- ⬜ 世界生成完整推理代码（WorldNav + World Composition）
+- ⬜ 全景生成（HY-Pano 2.0）模型和代码 — 可使用 [HunyuanWorld 1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0) 作为临时替代
+- ⬜ 世界扩展（WorldStereo 2.0）模型和代码 — 可使用 [WorldStereo](https://github.com/FuchengSu/WorldStereo) 作为临时替代
+
+
+## 🎁 模型库
+
+### 世界重建 — WorldMirror 系列
+
+| 模型 | 描述 | 参数量 | 日期 | Hugging Face |
+|------|------|--------|------|--------------|
+| WorldMirror 2.0 | 多视图 / 视频 &rarr; 3D 重建 | ~1.2B | 2026 | [下载](https://huggingface.co/tencent/HY-World-2.0/tree/main/HY-WorldMirror-2.0) |
+| WorldMirror 1.0 | 多视图 / 视频 &rarr; 3D 重建（旧版） | ~1.2B | 2025 | [下载](https://huggingface.co/tencent/HunyuanWorld-Mirror/tree/main) |
+
+### 全景生成
+
+| 模型 | 描述 | 参数量 | 日期 | Hugging Face |
+|------|------|--------|------|--------------|
+| HY-PanoGen | 文本 / 图像 &rarr; 360° 全景 | — | 即将发布 | — |
+
+### 世界生成
+
+| 模型            | 描述 | 参数量 | 日期 | Hugging Face |
+|-----------------|------|--------|------|--------------|
+| WorldStereo 2.0 | 全景 &rarr; 可导航 3DGS 世界 | — | 即将发布 | — |
+
+我们建议参考我们之前的工作 [WorldStereo](https://github.com/FuchengSu/WorldStereo) 和 [WorldMirror](https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror)，以了解世界生成和重建的背景知识。
+
+## 🤗 快速开始
+
+### 安装依赖
+
+我们建议使用 CUDA 12.4 进行安装。
+
+```bash
+# 1. 克隆仓库
+git clone https://github.com/Tencent-Hunyuan/HY-World-2.0
+cd HY-World-2.0
+
+# 2. 创建 conda 环境
+conda create -n hyworld2 python=3.10
+conda activate hyworld2
+
+# 3. 安装 PyTorch（CUDA 12.4）
+pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
+
+# 4. 安装依赖
+pip install -r requirements.txt
+
+# 5. 安装 FlashAttention
+# （推荐）安装 FlashAttention-3
+git clone https://github.com/Dao-AILab/flash-attention.git
+cd flash-attention/hopper
+python setup.py install
+cd ../../
+rm -rf flash-attention
+
+# 也可以使用更简单的 FlashAttention-2 安装方式
+pip install flash-attn --no-build-isolation
+```
+
+### 代码使用 — 全景生成（HY-Pano-2）
+
+*即将发布。*
+
+### 代码使用 — 世界生成（WorldNav、WorldStereo-2 和 3DGS）
+
+*即将发布。*
+
+**我们建议参考之前的工作 [WorldStereo](https://github.com/FuchengSu/WorldStereo)，作为 WorldStereo-2 的开源预览版本。**
+
+### 代码使用 — WorldMirror 2.0
+WorldMirror 2.0 支持以下使用方式：
+
+- [代码使用](#代码使用--worldmirror-20)
+- [Gradio 应用](#gradio-应用--worldmirror-20)
+
+我们提供了类似 `diffusers` 的 Python API。模型权重将在首次运行时自动从 Hugging Face 下载。
+
+```python
+from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
+
+pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
+result = pipeline('path/to/images')
+```
+
+**使用先验注入（相机位姿和深度）：**
+
+```python
+result = pipeline(
+    'path/to/images',
+    prior_cam_path='path/to/prior_camera.json',
+    prior_depth_path='path/to/prior_depth/',
+)
+```
+
+> 关于相机/深度先验的详细格式和准备方法，请参阅[先验准备指南](DOCUMENTATION_zh.md#先验注入)。
+
+**命令行：**
+
+```bash
+# 单卡推理
+python -m hyworld2.worldrecon.pipeline --input_path path/to/images
+
+# 多卡推理
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.pipeline \
+    --input_path path/to/images \
+    --use_fsdp --enable_bf16
+```
+
+> **重要提示：** 在多卡模式下，输入图像数量必须 **>= GPU 数量**。例如，使用 `--nproc_per_node=8` 时，需要提供至少 8 张图像。
+
+### Gradio 应用 — WorldMirror 2.0
+
+我们提供了一个交互式 [Gradio](https://www.gradio.app/) Web 演示。上传图像或视频，即可在浏览器中可视化 3DGS、点云、深度图、法线图和相机参数。
+
+```bash
+# 单卡
+python -m hyworld2.worldrecon.gradio_app
+
+# 多卡
+torchrun --nproc_per_node=2 -m hyworld2.worldrecon.gradio_app \
+    --use_fsdp --enable_bf16
+```
+
+关于 Gradio 应用的完整参数列表（端口、分享、本地检查点等），请参阅 [DOCUMENTATION_zh.md](DOCUMENTATION_zh.md#gradio-应用)。
+
+
+
+## 🔮 性能表现
+
+完整的基准测试结果请参阅[技术报告](https://3d-models.hunyuan.tencent.com/world/)。
+
+### WorldStereo 2.0 — 相机控制
+
+<table>
+  <thead>
+    <tr>
+      <th rowspan="2">方法</th>
+      <th colspan="3" align="center">相机指标</th>
+      <th colspan="4" align="center">视觉质量</th>
+    </tr>
+    <tr>
+      <th>RotErr ↓</th><th>TransErr ↓</th><th>ATE ↓</th>
+      <th>Q-Align ↑</th><th>CLIP-IQA+ ↑</th><th>Laion-Aes ↑</th><th>CLIP-I ↑</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr><td>SEVA</td><td>1.690</td><td>1.578</td><td>2.879</td><td>3.232</td><td>0.479</td><td>4.623</td><td>77.16</td></tr>
+    <tr><td>Gen3C</td><td>0.944</td><td>1.580</td><td>2.789</td><td>3.353</td><td>0.489</td><td>4.863</td><td>82.33</td></tr>
+    <tr><td>WorldStereo</td><td>0.762</td><td>1.245</td><td>2.141</td><td>4.149</td><td><b>0.547</b></td><td>5.257</td><td>89.05</td></tr>
+    <tr><td><b>WorldStereo 2.0</b></td><td><b>0.492</b></td><td><b>0.968</b></td><td><b>1.768</b></td><td><b>4.205</b></td><td>0.544</td><td><b>5.266</b></td><td><b>89.43</b></td></tr>
+  </tbody>
+</table>
+
+
+### WorldStereo 2.0 — 基于单帧输入的生成式重建
+
+<table>
+  <thead>
+    <tr>
+      <th rowspan="2">Methods</th>
+      <th colspan="4">Tanks-and-Temples</th>
+      <th colspan="4">MipNeRF360</th>
+    </tr>
+    <tr>
+      <th>Precision ↑</th>
+      <th>Recall ↑</th>
+      <th>F1-Score ↑</th>
+      <th>AUC ↑</th>
+      <th>Precision ↑</th>
+      <th>Recall ↑</th>
+      <th>F1-Score ↑</th>
+      <th>AUC ↑</th>
+    </tr>
+  </thead>
+  <tbody align="center">
+    <tr>
+      <td align="left">SEVA</td>
+      <td>33.59</td>
+      <td>35.34</td>
+      <td>36.73</td>
+      <td>51.03</td>
+      <td>22.38</td>
+      <td>55.63</td>
+      <td>28.75</td>
+      <td>46.81</td>
+    </tr>
+    <tr>
+      <td align="left">Gen3C</td>
+      <td><u>46.73</u></td>
+      <td>25.51</td>
+      <td>31.24</td>
+      <td>42.44</td>
+      <td>23.28</td>
+      <td><strong>75.37</strong></td>
+      <td>35.26</td>
+      <td>52.10</td>
+    </tr>
+    <tr>
+      <td align="left">Lyra</td>
+      <td><strong>50.38</strong></td>
+      <td>28.67</td>
+      <td>32.54</td>
+      <td>43.05</td>
+      <td>30.02</td>
+      <td>58.60</td>
+      <td>36.05</td>
+      <td>49.89</td>
+    </tr>
+    <tr>
+      <td align="left">FlashWorld</td>
+      <td>26.58</td>
+      <td>20.72</td>
+      <td>22.29</td>
+      <td>30.45</td>
+      <td>35.97</td>
+      <td>53.77</td>
+      <td>42.60</td>
+      <td>53.86</td>
+    </tr>
+    <tr>
+      <td align="left">WorldStereo 2.0</td>
+      <td>43.62</td>
+      <td><u>41.02</u></td>
+      <td><u>41.43</u></td>
+      <td><u>58.19</u></td>
+      <td><strong>43.19</strong></td>
+      <td><u>65.32</u></td>
+      <td><strong>51.27</strong></td>
+      <td><strong>65.79</strong></td>
+    </tr>
+    <tr>
+      <td align="left">WorldStereo 2.0 (DMD)</td>
+      <td>40.41</td>
+      <td><strong>44.41</strong></td>
+      <td><strong>43.16</strong></td>
+      <td><strong>60.09</strong></td>
+      <td><u>42.34</u></td>
+      <td>64.83</td>
+      <td><u>50.52</u></td>
+      <td><u>65.64</u></td>
+    </tr>
+  </tbody>
+</table>
+
+### WorldMirror 2.0 — 点云重建
+
+**在 7-Scenes、NRGBD 和 DTU 上的点图重建。** 我们报告了 WorldMirror 在不同输入配置下的平均精度和完整度。**加粗**为最优结果。"L / M / H" 分别代表低 / 中 / 高推理分辨率。"+ all priors" 表示同时注入相机外参、相机内参和深度先验。
+
+<table>
+  <thead>
+    <tr>
+      <th rowspan="2">方法</th>
+      <th colspan="2" align="center">7-Scenes <sub>(场景)</sub></th>
+      <th colspan="2" align="center">NRGBD <sub>(场景)</sub></th>
+      <th colspan="2" align="center">DTU <sub>(物体)</sub></th>
+    </tr>
+    <tr>
+      <th>Acc. ↓</th><th>Comp. ↓</th>
+      <th>Acc. ↓</th><th>Comp. ↓</th>
+      <th>Acc. ↓</th><th>Comp. ↓</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr><td colspan="7"><em>WorldMirror 1.0</em></td></tr>
+    <tr><td>&nbsp;&nbsp;L</td><td>0.043</td><td>0.055</td><td>0.046</td><td>0.049</td><td>1.476</td><td>1.768</td></tr>
+    <tr><td>&nbsp;&nbsp;L + all priors</td><td>0.021</td><td>0.026</td><td>0.022</td><td>0.020</td><td>1.347</td><td>1.392</td></tr>
+    <tr><td>&nbsp;&nbsp;M</td><td>0.043</td><td>0.049</td><td>0.041</td><td>0.045</td><td>1.017</td><td>1.780</td></tr>
+    <tr><td>&nbsp;&nbsp;M + all priors</td><td>0.018</td><td>0.023</td><td>0.016</td><td>0.014</td><td>0.735</td><td>0.935</td></tr>
+    <tr><td>&nbsp;&nbsp;H</td><td>0.079</td><td>0.087</td><td>0.077</td><td>0.093</td><td>2.271</td><td>2.113</td></tr>
+    <tr><td>&nbsp;&nbsp;H + all priors</td><td>0.042</td><td>0.041</td><td>0.078</td><td>0.082</td><td>1.773</td><td>1.478</td></tr>
+    <tr><td colspan="7"></td></tr>
+    <tr><td colspan="7"><em>WorldMirror 2.0</em></td></tr>
+    <tr><td>&nbsp;&nbsp;L</td><td>0.041</td><td>0.052</td><td>0.047</td><td>0.058</td><td>1.352</td><td>2.009</td></tr>
+    <tr><td>&nbsp;&nbsp;L + all priors</td><td>0.019</td><td>0.024</td><td>0.017</td><td>0.015</td><td>1.100</td><td>1.201</td></tr>
+    <tr><td>&nbsp;&nbsp;M</td><td>0.033</td><td>0.046</td><td>0.039</td><td>0.047</td><td>1.005</td><td>1.892</td></tr>
+    <tr><td>&nbsp;&nbsp;M + all priors</td><td>0.013</td><td>0.017</td><td><b>0.013</b></td><td><b>0.013</b></td><td>0.690</td><td>0.876</td></tr>
+    <tr><td>&nbsp;&nbsp;H</td><td>0.037</td><td>0.040</td><td>0.046</td><td>0.053</td><td>0.845</td><td>1.904</td></tr>
+    <tr><td>&nbsp;&nbsp;<b>H + all priors</b></td><td><b>0.012</b></td><td><b>0.016</b></td><td>0.015</td><td>0.016</td><td><b>0.554</b></td><td><b>0.771</b></td></tr>
+  </tbody>
+</table>
+ 
+### WorldMirror 2.0 — 先验对比
+
+**WorldMirror 与 Pow3R、MapAnything 在不同先验条件下的对比。** 结果为 7-Scenes、NRGBD 和 DTU 数据集上的平均值。Pow3R (pro) 指使用 Procrustes 对齐的原版 Pow3R。
+
+
+<p align="center">
+  <img src="assets/prior_comparison2_wm2.png" width="85%">
+</p>
+
+
+
+
+## 🎬 更多示例
+
+<table align="center" style="border: none;">
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_3.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_4.gif" width="100%"></td>
+  </tr>
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_5.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_6.gif" width="100%"></td>
+  </tr>
+  <tr>
+    <td align="center" width="50%"><img src="assets/screenshot_9.gif" width="100%"></td>
+    <td align="center" width="50%"><img src="assets/screenshot_10.gif" width="100%"></td>
+  </tr>
+</table>
+
+
+## 📖 文档
+
+详细的使用指南、参数参考、输出格式说明和先验注入说明，请参阅 **[DOCUMENTATION_zh.md](DOCUMENTATION_zh.md)**。
+
+
+## 📚 引用
+
+如果您觉得 HunyuanWorld 2.0 对您的研究有帮助，请引用：
+
+```bibtex
+@article{hyworld22026,
+  title={HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating and Simulating 3D Worlds},
+  author={Tencent HY-World Team},
+  journal={arXiv preprint},
+  year={2026}
+}
+
+@article{hunyuanworld2025tencent,
+    title={HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels},
+    author={Team HunyuanWorld},
+    year={2025},
+    journal={arXiv preprint}
+}
+```
+
+
+## 📧 联系方式
+
+如有任何问题或反馈，请发送邮件至 tengfeiwang12@gmail.com。
+
+
+## 🙏 致谢
+
+我们衷心感谢 [HunyuanWorld 1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0)、[WorldMirror](https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror)、[WorldPlay](https://github.com/Tencent-Hunyuan/HY-WorldPlay)、[WorldStereo](https://github.com/FuchengSu/WorldStereo)、[HunyuanImage](https://github.com/Tencent-Hunyuan/HunyuanImage-3.0) 的杰出工作。

assets/hyworld2_en.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd227cfaf618716dd4d4c8a8a9d032947ac3cd60c1adc6771bd7a066069b94fd
+size 20726000