Upload folder using huggingface_hub
Browse files- .gitattributes +3 -0
- README.md +55 -3
- assets/example_output_dog_pose.mp4 +3 -0
- assets/example_output_high_plank.mp4 +3 -0
- assets/teaser.png +3 -0
- diffusion_pytorch_model.safetensors +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/example_output_dog_pose.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/example_output_high_plank.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/teaser.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,3 +1,55 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.1/css/all.min.css" integrity="sha512-DTOQO9RWCH3ppGqcWaEA1BIZOC6xxalwEsw9c2QQeAIftl+Vegovlnee1c9QX4TctnWMn13TZye+giMm8e2LwA==" crossorigin="anonymous" referrerpolicy="no-referrer" />
|
| 2 |
+
|
| 3 |
+
<h1 align="center">CoMoVi: Co-Generation of 3D Human Motions<br>and Realistic Videos</h1>
|
| 4 |
+
|
| 5 |
+
<p align="center">
|
| 6 |
+
<a href="https://afterjourney00.github.io/" target="_blank">Chengfeng Zhao</a><sup>1</sup>,
|
| 7 |
+
<a href="https://github.com/Samir1110" target="_blank">Jiazhi Shu</a><sup>2</sup>,
|
| 8 |
+
<a href="https://knoxzhao.github.io/" target="_blank">Yubo Zhao</a><sup>1</sup>,
|
| 9 |
+
<a href="https://scholar.google.com/citations?hl=en&user=nhbSplwAAAAJ" target="_blank">Tianyu Huang</a><sup>3</sup>,
|
| 10 |
+
<a href="https://scholar.google.com/citations?hl=en&user=nhbSplwAAAAJ" target="_blank">Jiahao Lu</a><sup>1</sup>,
|
| 11 |
+
<br>
|
| 12 |
+
<a href="https://scholar.google.com/citations?hl=en&user=nhbSplwAAAAJ" target="_blank">Zekai Gu</a><sup>1</sup>,
|
| 13 |
+
<a href="https://scholar.google.com/citations?hl=en&user=nhbSplwAAAAJ" target="_blank">Chengwei Ren</a><sup>1</sup>,
|
| 14 |
+
<a href="https://frank-zy-dou.github.io/" target="_blank">Zhiyang Dou</a><sup>4</sup>,
|
| 15 |
+
<a href="https://chingswy.github.io/" target="_blank">Qing Shuai</a><sup>5</sup>,
|
| 16 |
+
<a href="https://liuyuan-pal.github.io/" target="_blank">Yuan Liu</a><sup>1 <i class="far fa-envelope"></i></sup>
|
| 17 |
+
</p>
|
| 18 |
+
<p align="center">
|
| 19 |
+
<sup>1</sup>HKUST
|
| 20 |
+
<sup>2</sup>SCUT
|
| 21 |
+
<sup>3</sup>CUHK
|
| 22 |
+
<sup>4</sup>MIT
|
| 23 |
+
<sup>5</sup>ZJU
|
| 24 |
+
<br>
|
| 25 |
+
<i><sup><i class="far fa-envelope"></i></sup> Corresponding author</i>
|
| 26 |
+
</p>
|
| 27 |
+
<p align="center">
|
| 28 |
+
<a href="https://igl-hkust.github.io/CoMoVi/"><img src='https://img.shields.io/badge/arXiv-Paper-red?logo=arxiv&logoColor=white' alt='arXiv'></a>
|
| 29 |
+
<a href='https://igl-hkust.github.io/CoMoVi/'><img src='https://img.shields.io/badge/Project_Page-Website-green?logo=googlechrome&logoColor=white' alt='Project Page'></a>
|
| 30 |
+
<a href='https://huggingface.co/datasets/AfterJourney/CoMoVi-50K'><img src='https://img.shields.io/badge/Hugging%20Face-Dataset-yellow?logo=huggingface' alt='Dataset'></a>
|
| 31 |
+
</p>
|
| 32 |
+
|
| 33 |
+
<div align="center">
|
| 34 |
+
<img width="900px" src="./assets/teaser.png"/>
|
| 35 |
+
</div>
|
| 36 |
+
|
| 37 |
+
## <i class="fa-brands fa-github"></i> [GitHub](https://github.com/IGL-HKUST/CoMoVi)
|
| 38 |
+
|
| 39 |
+
## Acknowledgments
|
| 40 |
+
|
| 41 |
+
Thanks to the following work that we refer to and benefit from:
|
| 42 |
+
- [VideoX-Fun](https://github.com/aigc-apps/VideoX-Fun): the video generation model training framework;
|
| 43 |
+
- [CameraHMR](https://github.com/pixelite1201/CameraHMR/): the excellent SMPL estimation for pseudo labels;
|
| 44 |
+
- [Champ](https://github.com/fudan-generative-vision/champ): the data processing pipeline
|
| 45 |
+
|
| 46 |
+
## Citation
|
| 47 |
+
|
| 48 |
+
```bibtex
|
| 49 |
+
@article{zhao2026comovi,
|
| 50 |
+
title={CoMoVi: Co-Generation of 3D Human Motions and Realistic Videos},
|
| 51 |
+
author={Zhao, Chengfeng and Shu, Jiazhi and Zhao, Yubo and Huang, Tianyu and Lu, Jiahao and Gu, Zekai and Ren, Chengwei and Dou, Zhiyang and Shuai, Qing and Liu, Yuan},
|
| 52 |
+
journal={arXiv preprint arXiv:2601.10632},
|
| 53 |
+
year={2026}
|
| 54 |
+
}
|
| 55 |
+
```
|
assets/example_output_dog_pose.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:371cacbd2ba292a201f7de49978ba8e41df090bfce47fbd4558b3ce3920a12a6
|
| 3 |
+
size 440869
|
assets/example_output_high_plank.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b663f3554a31dc732fba259b3b2afeda5c1309d3f26109ad335e04f37eae08e3
|
| 3 |
+
size 378168
|
assets/teaser.png
ADDED
|
Git LFS Details
|
diffusion_pytorch_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:55e5494933b16668683f2d38989c3688e985ff45118c67528353a536482eba2d
|
| 3 |
+
size 22165148424
|