Add sections for Introduction and Framework to the README.md

.gitattributes

@@ -32,4 +32,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.xz filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -8,3 +8,52 @@ base_model:
 tags:
 - joint-audio-video-generation
 ---
+
+<h1 align="center">Hallo-Live</h1>
+<!-- <h1 align="center">Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar</h1> -->
+
+<div align='center'>
+<a href="https://github.com/chunyu-li" target="_blank">Chunyu Li</a><sup>1,2,*</sup> &emsp;
+<a href="https://github.com/fudan-generative-vision/Hallo-Live" target="_blank">Jiaye Li</a><sup>2,*</sup> &emsp;
+<a href="https://github.com/fudan-generative-vision/Hallo-Live" target="_blank">Ruiqiao Mei</a><sup>2</sup> &emsp;
+<a href="https://github.com/fudan-generative-vision/Hallo-Live" target="_blank">Haoyuan Xia</a><sup>1,3</sup>
+</div>
+<div align='center'>
+<a href="http://zhuhao.cc/home/" target="_blank">Hao Zhu</a><sup>4</sup> &emsp;
+<a href="https://jingdongwang2017.github.io/" target="_blank">Jingdong Wang</a><sup>5</sup> &emsp;
+<a href="https://sites.google.com/site/zhusiyucs/home" target="_blank">Siyu Zhu</a><sup>1,2,&dagger;</sup>
+</div>
+
+<br>
+
+<div align='center'>
+<sup>1</sup>Shanghai Innovation Institute &emsp;
+<sup>2</sup>Fudan University
+</div>
+<div align='center'>
+<sup>3</sup>University of Science and Technology of China &emsp;
+<sup>4</sup>Nanjing University &emsp;
+<sup>5</sup>Baidu
+</div>
+
+<br>
+
+<div align="center">
+
+[![Paper](https://img.shields.io/badge/arXiv-2604.23632-b31b1b.svg)](https://arxiv.org/abs/2604.23632)
+[![arXiv](https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Model-yellow)](https://huggingface.co/fudan-generative-ai/Hallo-Live)
+[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+</div>
+
+## 📖 Introduction
+
+We present *Hallo-Live*, a real-time text-driven joint audio-video avatar generation framework. The method adopts a causal dual-stream DiT model to generate synchronized avatar video and speech in a streaming manner. *Hallo-Live* reaches **20.38 FPS** with **0.94 s latency** on two NVIDIA H200 GPUs, while preserving strong lip-sync accuracy, visual fidelity, and speech quality.
+
+## 🏗️ Framework
+
+<p align="center">
+<img src="assets/framework.png" width=100%>
+<p>
+
+The framework of *Hallo-Live*. **Top left**: Stage I training adapts a pretrained dual-stream DiT to the streaming setting using cross-modal future-expanding block-causal mask. **Bottom left**: Stage II training performs autoregressive self-rollout with the audio-video KV cache and optimizes the generated trajectory with reward-weighted dual-stream DMD. **Right**: Each causal fusion block in the dual-stream DiT consists of cross-modal attention between the video and audio streams, where the block-causal masks are utilized in Stage I ODE initialization, and KV cache is maintained for Stage II self-rollout and streaming inference.