Instructions to use Jackrong/Qwopus3.6-27B-v2-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jackrong/Qwopus3.6-27B-v2-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Jackrong/Qwopus3.6-27B-v2-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Jackrong/Qwopus3.6-27B-v2-FP8")
model = AutoModelForImageTextToText.from_pretrained("Jackrong/Qwopus3.6-27B-v2-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Jackrong/Qwopus3.6-27B-v2-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jackrong/Qwopus3.6-27B-v2-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jackrong/Qwopus3.6-27B-v2-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Jackrong/Qwopus3.6-27B-v2-FP8

SGLang

How to use Jackrong/Qwopus3.6-27B-v2-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jackrong/Qwopus3.6-27B-v2-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jackrong/Qwopus3.6-27B-v2-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jackrong/Qwopus3.6-27B-v2-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jackrong/Qwopus3.6-27B-v2-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Studio new

How to use Jackrong/Qwopus3.6-27B-v2-FP8 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jackrong/Qwopus3.6-27B-v2-FP8 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jackrong/Qwopus3.6-27B-v2-FP8 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Jackrong/Qwopus3.6-27B-v2-FP8 to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Jackrong/Qwopus3.6-27B-v2-FP8",
    max_seq_length=2048,
)

Docker Model Runner
How to use Jackrong/Qwopus3.6-27B-v2-FP8 with Docker Model Runner:
```
docker model run hf.co/Jackrong/Qwopus3.6-27B-v2-FP8
```

hello,The model's output is garbled.

by zhao198300 - opened about 15 hours ago

Discussion

zhao198300

about 15 hours ago

Did I do something wrong?

vllm run command ：
VLLM_ATTENTION_BACKEND=FLASH_ATTN
vllm serve
--model /models/qwopus3.6-27b-v2-fp8
--host 0.0.0.0
--port 8080
--kv_cache_dtype fp8
--tool-call-parser qwen3_coder
--reasoning-parser qwen3
--enable-auto-tool-choice
--tensor-parallel-size 2
--max-model-len 262144
--gpu-memory-utilization 0.9046
--trust-remote-code
--compilation-config '{"cudagraph_mode": "PIECEWISE"}'
--default-chat-template-kwargs '{"enable_thinking": true, "preserve_thinking":true}'
--max-num-seqs 2
--compilation_config.mode VLLM_COMPILE
--enable-prefix-caching
--enable-chunked-prefill
--served-model-name qwen3.6-27b
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
--max-num-batched-tokens 16384
--attention-backend FLASHINFER

maglat

about 13 hours ago

For me it isnt working as well. Thinking for infinity :)

Jackrong

Owner about 11 hours ago

Hi there, I am terribly sorry for the inconvenience!

I will look into this model issue immediately. For context, all of my testing was conducted using the transformers library, where it was able to output normally. I haven't used vllm myself, so there might be some compatibility or configuration issues.

Please bear with me for a moment while I run some tests and checks. I will update you here as soon as I find anything!

Jackrong

Owner about 7 hours ago

Did I do something wrong?

vllm run command ：
VLLM_ATTENTION_BACKEND=FLASH_ATTN
vllm serve
--model /models/qwopus3.6-27b-v2-fp8
--host 0.0.0.0
--port 8080
--kv_cache_dtype fp8
--tool-call-parser qwen3_coder
--reasoning-parser qwen3
--enable-auto-tool-choice
--tensor-parallel-size 2
--max-model-len 262144
--gpu-memory-utilization 0.9046
--trust-remote-code
--compilation-config '{"cudagraph_mode": "PIECEWISE"}'
--default-chat-template-kwargs '{"enable_thinking": true, "preserve_thinking":true}'
--max-num-seqs 2
--compilation_config.mode VLLM_COMPILE
--enable-prefix-caching
--enable-chunked-prefill
--served-model-name qwen3.6-27b
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
--max-num-batched-tokens 16384
--attention-backend FLASHINFER

For me it isnt working as well. Thinking for infinity :)

Hi everyone！

Thank you so much for your feedback!

I have now fixed this issue and updated the model weights, model card, and recommended test configurations.

I’m terribly sorry for the inconvenience caused earlier. Please pull the latest model and give it another try!

maglat

about 3 hours ago

Many thanks for the very fast reaction. There have to be now way you have to be sorry. We need to be thankful for everything you are doing :)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment