Add image-to-3d pipeline tag and improve model card (#1)
Browse files- Add image-to-3d pipeline tag and improve model card (45fafc48f156a48954cb4d3a2805e9df39c564a1)
Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -1,117 +1,108 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: other
|
| 3 |
-
license_name: pixal3d-license
|
| 4 |
-
license_link: LICENSE
|
| 5 |
-
extra_gated_eu_disallowed: true
|
| 6 |
-
--
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
<a href="https://
|
| 25 |
-
<a href="https://huggingface.co/TencentARC/Pixal3D"><img src=https://img.shields.io/badge/%F0%9F%A4%97%
|
| 26 |
-
<a href="https://
|
| 27 |
-
</
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
- **May 2026**: Release
|
| 37 |
-
- **
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
|
| 43 |
-
|
|
| 44 |
-
| `
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
```
|
| 109 |
-
@article{li2026pixal3d,
|
| 110 |
-
title = {Pixal3D: Pixel-Aligned 3D Generation from Images},
|
| 111 |
-
author = {Li, Dong-Yang and Zhao, Wang and Chen, Yuxin and Hu, Wenbo and Guo, Meng-Hao and Zhang, Fang-Lue and Shan, Ying and Hu, Shi-Min},
|
| 112 |
-
journal = {arXiv preprint arXiv:2605.10922},
|
| 113 |
-
year = {2026}
|
| 114 |
-
}
|
| 115 |
-
```
|
| 116 |
-
|
| 117 |
-
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: pixal3d-license
|
| 4 |
+
license_link: LICENSE
|
| 5 |
+
extra_gated_eu_disallowed: true
|
| 6 |
+
pipeline_tag: image-to-3d
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
<div align="center">
|
| 10 |
+
|
| 11 |
+
# Pixal3D: Pixel-Aligned 3D Generation from Images
|
| 12 |
+
|
| 13 |
+
<h3>SIGGRAPH 2026</h3>
|
| 14 |
+
|
| 15 |
+
[Dong-Yang Li](https://ldyang694.github.io/)¹ · [Wang Zhao](https://thuzhaowang.github.io/)²* · [Yuxin Chen](https://orcid.org/0000-0002-7854-1072)² · [Wenbo Hu](https://wbhu.github.io/)² · [Meng-Hao Guo](https://menghaoguo.github.io/)¹ · [Fang-Lue Zhang](https://fanglue.github.io/)³ · [Ying Shan](https://www.linkedin.com/in/YingShanProfile)² · [Shi-Min Hu](https://cg.cs.tsinghua.edu.cn/shimin.htm)¹✉
|
| 16 |
+
|
| 17 |
+
¹Tsinghua University (BNRist) ²Tencent ARC Lab ³Victoria University of Wellington
|
| 18 |
+
|
| 19 |
+
*Project lead ✉Corresponding author
|
| 20 |
+
|
| 21 |
+
</div>
|
| 22 |
+
|
| 23 |
+
<div align="center">
|
| 24 |
+
<a href="https://ldyang694.github.io/projects/pixal3d/"><img src=https://img.shields.io/badge/Project%20Page-333399.svg?logo=googlehome height=22px></a>
|
| 25 |
+
<a href="https://huggingface.co/spaces/TencentARC/Pixal3D"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Demo-276cb4.svg height=22px></a>
|
| 26 |
+
<a href="https://github.com/TencentARC/Pixal3D"><img src=https://img.shields.io/badge/Code-Github-black.svg?logo=github height=22px></a>
|
| 27 |
+
<a href="https://arxiv.org/abs/2605.10922"><img src=https://img.shields.io/badge/Arxiv-b5212f.svg?logo=arxiv height=22px></a>
|
| 28 |
+
</div>
|
| 29 |
+
|
| 30 |
+
**Pixal3D** generates high-fidelity 3D assets from a single image. Unlike previous methods that loosely inject image features via attention, Pixal3D explicitly lifts pixel features into 3D through back-projection, establishing direct pixel-to-3D correspondences. This enables near-reconstruction-level fidelity with detailed geometry and PBR textures.
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
|
| 34 |
+
## ✨ News
|
| 35 |
+
|
| 36 |
+
- **May 2026**: Release the improved version based on [Trellis.2](https://github.com/microsoft/TRELLIS.2) backbone. 💪
|
| 37 |
+
- **May 2026**: Release inference code and online demo. 🤗
|
| 38 |
+
- **Apr 2026**: Our paper is accepted to SIGGRAPH 2026! 🎉
|
| 39 |
+
|
| 40 |
+
## 📌 Branches
|
| 41 |
+
|
| 42 |
+
| Branch | Description |
|
| 43 |
+
|--------|-------------|
|
| 44 |
+
| `main` | **Latest version** — improved implementation based on [Trellis.2](https://github.com/microsoft/TRELLIS.2) backbone with better performance. |
|
| 45 |
+
| `paper` | **Paper version** — original implementation based on [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2), corresponding to results reported in our SIGGRAPH 2026 paper. |
|
| 46 |
+
|
| 47 |
+
> If you want to reproduce the results in our paper, please switch to the `paper` branch.
|
| 48 |
+
|
| 49 |
+
## 🎮 Try It Online
|
| 50 |
+
|
| 51 |
+
You can try Pixal3D directly in your browser without any installation via our Hugging Face Gradio demo:
|
| 52 |
+
|
| 53 |
+
👉 [**Launch Demo**](https://huggingface.co/spaces/TencentARC/Pixal3D)
|
| 54 |
+
|
| 55 |
+
## 🚀 Getting Started
|
| 56 |
+
|
| 57 |
+
### Installation
|
| 58 |
+
|
| 59 |
+
#### Step 1: Follow TRELLIS.2 Installation
|
| 60 |
+
|
| 61 |
+
Please first follow the installation guide of [TRELLIS.2](https://github.com/microsoft/TRELLIS.2) to set up the base environment.
|
| 62 |
+
|
| 63 |
+
#### Step 2: Install Additional Dependencies
|
| 64 |
+
|
| 65 |
+
```bash
|
| 66 |
+
pip install -r requirements.txt
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
#### Step 3: Install utils3d
|
| 70 |
+
|
| 71 |
+
```bash
|
| 72 |
+
pip install https://github.com/LDYang694/Storages/releases/download/20260430/utils3d-0.0.2-py3-none-any.whl
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
### Usage
|
| 76 |
+
|
| 77 |
+
#### Inference
|
| 78 |
+
|
| 79 |
+
Generate a GLB mesh from a single image:
|
| 80 |
+
|
| 81 |
+
```bash
|
| 82 |
+
python inference.py --image assets/test_image/0.png --output ./output.glb
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
### Web Demo
|
| 86 |
+
|
| 87 |
+
We provide a Gradio web demo for Pixal3D, which allows you to generate 3D meshes from images interactively.
|
| 88 |
+
|
| 89 |
+
```bash
|
| 90 |
+
python app.py
|
| 91 |
+
```
|
| 92 |
+
|
| 93 |
+
## 🤗 Acknowledgements
|
| 94 |
+
|
| 95 |
+
This project is heavily built upon [Trellis.2](https://github.com/microsoft/TRELLIS.2) and [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2). We also thank the following repos for their great contributions: [Trellis](https://github.com/microsoft/TRELLIS).
|
| 96 |
+
|
| 97 |
+
## 📄 Citation
|
| 98 |
+
|
| 99 |
+
If you find this work useful, please consider citing:
|
| 100 |
+
|
| 101 |
+
```bibtex
|
| 102 |
+
@article{li2026pixal3d,
|
| 103 |
+
title = {Pixal3D: Pixel-Aligned 3D Generation from Images},
|
| 104 |
+
author = {Li, Dong-Yang and Zhao, Wang and Chen, Yuxin and Hu, Wenbo and Guo, Meng-Hao and Zhang, Fang-Lue and Shan, Ying and Hu, Shi-Min},
|
| 105 |
+
journal = {arXiv preprint arXiv:2605.10922},
|
| 106 |
+
year = {2026}
|
| 107 |
+
}
|
| 108 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|