Instructions to use shawnw3i/Qwen3-Reranker-0.6B-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use shawnw3i/Qwen3-Reranker-0.6B-ONNX with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="shawnw3i/Qwen3-Reranker-0.6B-ONNX")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("shawnw3i/Qwen3-Reranker-0.6B-ONNX")
model = AutoModelForCausalLM.from_pretrained("shawnw3i/Qwen3-Reranker-0.6B-ONNX")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use shawnw3i/Qwen3-Reranker-0.6B-ONNX with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shawnw3i/Qwen3-Reranker-0.6B-ONNX"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shawnw3i/Qwen3-Reranker-0.6B-ONNX",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/shawnw3i/Qwen3-Reranker-0.6B-ONNX

SGLang

How to use shawnw3i/Qwen3-Reranker-0.6B-ONNX with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "shawnw3i/Qwen3-Reranker-0.6B-ONNX" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shawnw3i/Qwen3-Reranker-0.6B-ONNX",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "shawnw3i/Qwen3-Reranker-0.6B-ONNX" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shawnw3i/Qwen3-Reranker-0.6B-ONNX",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use shawnw3i/Qwen3-Reranker-0.6B-ONNX with Docker Model Runner:
```
docker model run hf.co/shawnw3i/Qwen3-Reranker-0.6B-ONNX
```

Qwen3-Reranker-0.6B-ONNX

Commit History

Update README.md

21fb0d0
verified

shawnw3i commited on 5 days ago

Update README.md

920d526
verified

zhiqing commited on Jun 25, 2025

Update README.md

ad5f45a
verified

zhiqing commited on Jun 25, 2025

Update README.md

15af9d6
verified

zhiqing commited on Jun 20, 2025

Fix export error

13be2c7
verified

zhiqing commited on Jun 20, 2025

Update README.md

74fc010
verified

zhiqing commited on Jun 18, 2025

Fix error

ea7555d
verified

zhiqing commited on Jun 18, 2025

Fix export error

8effd47
verified

zhiqing commited on Jun 18, 2025

Create README.md

2405522
verified

zhiqing commited on Jun 9, 2025

Upload folder using huggingface_hub

0d0e81b
verified

zhiqing commited on Jun 9, 2025

initial commit

1245089
verified

zhiqing commited on Jun 9, 2025

Commit History

Update README.md 21fb0d0 verified

Update README.md 920d526 verified

Update README.md ad5f45a verified

Update README.md 15af9d6 verified

Fix export error 13be2c7 verified

Update README.md 74fc010 verified

Fix error ea7555d verified

Fix export error 8effd47 verified

Create README.md 2405522 verified

Upload folder using huggingface_hub 0d0e81b verified

initial commit 1245089 verified

Update README.md

21fb0d0
verified

Update README.md

920d526
verified

Update README.md

ad5f45a
verified

Update README.md

15af9d6
verified

Fix export error

13be2c7
verified

Update README.md

74fc010
verified

Fix error

ea7555d
verified

Fix export error

8effd47
verified

Create README.md

2405522
verified

Upload folder using huggingface_hub

0d0e81b
verified

initial commit

1245089
verified