Image-Text-to-Text
Transformers
Safetensors
English
qwen3_vl
agent
image-generation
tool-use
visual-reasoning
self-distillation
grpo
reinforcement-learning
multimodal
qwen3-vl
conversational
Instructions to use MeiGen-AI/GenEvolve with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MeiGen-AI/GenEvolve with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="MeiGen-AI/GenEvolve") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("MeiGen-AI/GenEvolve") model = AutoModelForImageTextToText.from_pretrained("MeiGen-AI/GenEvolve") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use MeiGen-AI/GenEvolve with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MeiGen-AI/GenEvolve" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MeiGen-AI/GenEvolve", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/MeiGen-AI/GenEvolve
- SGLang
How to use MeiGen-AI/GenEvolve with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MeiGen-AI/GenEvolve" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MeiGen-AI/GenEvolve", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MeiGen-AI/GenEvolve" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MeiGen-AI/GenEvolve", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use MeiGen-AI/GenEvolve with Docker Model Runner:
docker model run hf.co/MeiGen-AI/GenEvolve
README: sync Quick Start with latest GitHub (separate Qwen-Image-Edit FastAPI service, MeiGen-AI/GenEvolve as MODEL_PATH, install order)
Browse files
README.md
CHANGED
|
@@ -111,27 +111,56 @@ For a user request, the agent samples a multi-turn trajectory of tool calls befo
|
|
| 111 |
|
| 112 |
## 🚀 Quick Start
|
| 113 |
|
| 114 |
-
The deployed checkpoint is the **student policy** — it consumes a user prompt and returns a JSON `gen_prompt + reference_images` program through a `<think>/<tool_call>/<answer>` loop. The end-to-end runtime (vLLM
|
|
|
|
|
|
|
| 115 |
|
| 116 |
```bash
|
| 117 |
git clone https://github.com/Ephemeral182/GenEvolve.git
|
| 118 |
cd GenEvolve
|
|
|
|
| 119 |
conda create -n genevolve python=3.11 -y && conda activate genevolve
|
|
|
|
| 120 |
pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128
|
| 121 |
-
pip install --no-build-isolation -r requirements.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
| 122 |
|
| 123 |
-
# Serve the
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
MODEL_PATH=MeiGen-AI/GenEvolve PORT=8000 TP=1 DP=8 bash scripts/serve_vllm.sh
|
|
|
|
| 125 |
|
| 126 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
export SERPER_API_KEY=<your_key> # required for search / image_search
|
| 128 |
-
export GOOGLE_API_KEY=<your_key> # only for
|
|
|
|
|
|
|
| 129 |
python examples/quickstart.py \
|
| 130 |
--backend nano-banana-pro \
|
| 131 |
--base-url http://localhost:8000/v1 \
|
| 132 |
--model GenEvolve \
|
| 133 |
--prompt "A 1990s travel-magazine cover of two backpackers in front of the Eiffel Tower at golden hour, the title \"PARIS\" in bold serif." \
|
| 134 |
--output paris.png
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 135 |
```
|
| 136 |
|
| 137 |
The agent's final `<answer>` is a JSON object:
|
|
|
|
| 111 |
|
| 112 |
## 🚀 Quick Start
|
| 113 |
|
| 114 |
+
The deployed checkpoint is the **student policy** — it consumes a user prompt and returns a JSON `gen_prompt + reference_images` program through a `<think>/<tool_call>/<answer>` loop. The end-to-end runtime (vLLM serving + agent loop + tools + Qwen/Nano renderers) lives in the [GitHub repo](https://github.com/Ephemeral182/GenEvolve); the snippet below mirrors its installation and usage.
|
| 115 |
+
|
| 116 |
+
### 1. Install the main GenEvolve runtime
|
| 117 |
|
| 118 |
```bash
|
| 119 |
git clone https://github.com/Ephemeral182/GenEvolve.git
|
| 120 |
cd GenEvolve
|
| 121 |
+
|
| 122 |
conda create -n genevolve python=3.11 -y && conda activate genevolve
|
| 123 |
+
pip install -U pip setuptools wheel packaging psutil ninja
|
| 124 |
pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128
|
| 125 |
+
pip install --no-build-isolation -r requirements.txt
|
| 126 |
+
pip install -e .
|
| 127 |
+
```
|
| 128 |
+
|
| 129 |
+
Qwen-Image-Edit rendering runs as a **separate FastAPI service** (kept out of the vLLM environment to avoid CUDA/diffusers conflicts). Set up that service from the GitHub README when you want to use `--backend qwen-image-edit-service`.
|
| 130 |
|
| 131 |
+
### 2. Serve the agent policy
|
| 132 |
+
|
| 133 |
+
```bash
|
| 134 |
+
# Single GPU / single replica.
|
| 135 |
+
MODEL_PATH=MeiGen-AI/GenEvolve PORT=8000 TP=1 DP=1 bash scripts/serve_vllm.sh
|
| 136 |
+
|
| 137 |
+
# Higher throughput on one 8-GPU node (8 replicas, 1 GPU each).
|
| 138 |
MODEL_PATH=MeiGen-AI/GenEvolve PORT=8000 TP=1 DP=8 bash scripts/serve_vllm.sh
|
| 139 |
+
```
|
| 140 |
|
| 141 |
+
`TP` shards one model replica across multiple GPUs; `DP` launches multiple replicas; total GPU usage is `TP × DP`.
|
| 142 |
+
|
| 143 |
+
### 3. End-to-end example
|
| 144 |
+
|
| 145 |
+
```bash
|
| 146 |
export SERPER_API_KEY=<your_key> # required for search / image_search
|
| 147 |
+
export GOOGLE_API_KEY=<your_key> # only for --backend nano-banana-pro
|
| 148 |
+
|
| 149 |
+
# Nano Banana Pro renderer
|
| 150 |
python examples/quickstart.py \
|
| 151 |
--backend nano-banana-pro \
|
| 152 |
--base-url http://localhost:8000/v1 \
|
| 153 |
--model GenEvolve \
|
| 154 |
--prompt "A 1990s travel-magazine cover of two backpackers in front of the Eiffel Tower at golden hour, the title \"PARIS\" in bold serif." \
|
| 155 |
--output paris.png
|
| 156 |
+
|
| 157 |
+
# Qwen-Image-Edit renderer (point at your Qwen-Image-Edit FastAPI service)
|
| 158 |
+
python examples/quickstart.py \
|
| 159 |
+
--backend qwen-image-edit-service \
|
| 160 |
+
--service-url http://your-qwen-service:8001 \
|
| 161 |
+
--base-url http://localhost:8000/v1 \
|
| 162 |
+
--model GenEvolve \
|
| 163 |
+
--output paris_qwen.png
|
| 164 |
```
|
| 165 |
|
| 166 |
The agent's final `<answer>` is a JSON object:
|