Instructions to use moonshotai/Kimi-K2.6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use moonshotai/Kimi-K2.6 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K2.6", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("moonshotai/Kimi-K2.6", trust_remote_code=True, dtype="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use moonshotai/Kimi-K2.6 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "moonshotai/Kimi-K2.6"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K2.6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/moonshotai/Kimi-K2.6

SGLang

How to use moonshotai/Kimi-K2.6 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "moonshotai/Kimi-K2.6" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K2.6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "moonshotai/Kimi-K2.6" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K2.6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use moonshotai/Kimi-K2.6 with Docker Model Runner:
```
docker model run hf.co/moonshotai/Kimi-K2.6
```

Kimi-K2.6

Commit History

Use uv (preferred) with --prerelease=allow for sglang install

06b359a
verified

JustinTong commited on 17 days ago

Use sglang>=0.5.10.post1 (drop [all] extra)

8a5354a
verified

JustinTong commited on 17 days ago

Update SGLang deployment guide: point to K2.6 cookbook and use stable release

b5632fb
verified

JustinTong commited on 17 days ago

fix: typo

2b2b88e
verified

FuryMartin commited on 17 days ago

Add evaluation results for HLE, GPQA, AIME, HMMT, SWE-Bench, and Terminal-Bench (#4)

d9cb81b

bigeagle

SaylorTwift HF Staff commited on 17 days ago

add tech blog

36d7872

weichu commited on 17 days ago

update footnote

e69e669
verified

FuryMartin commited on 17 days ago

Update README.md

20008c5
verified

bigmoyan commited on 17 days ago

Update README.md

4931c56
verified

bigeagle commited on 17 days ago

Add files using upload-large-folder tool

e0ee936

courage17340 commited on 23 days ago

initial commit

344a634
verified

courage17340 commited on 23 days ago

Commit History

Use uv (preferred) with --prerelease=allow for sglang install 06b359a verified

Use sglang>=0.5.10.post1 (drop [all] extra) 8a5354a verified

Update SGLang deployment guide: point to K2.6 cookbook and use stable release b5632fb verified

fix: typo 2b2b88e verified

Add evaluation results for HLE, GPQA, AIME, HMMT, SWE-Bench, and Terminal-Bench (#4) d9cb81b

add tech blog 36d7872

update footnote e69e669 verified

Update README.md 20008c5 verified

Update README.md 4931c56 verified

Add files using upload-large-folder tool e0ee936

initial commit 344a634 verified

Use uv (preferred) with --prerelease=allow for sglang install

06b359a
verified

Use sglang>=0.5.10.post1 (drop [all] extra)

8a5354a
verified

Update SGLang deployment guide: point to K2.6 cookbook and use stable release

b5632fb
verified

fix: typo

2b2b88e
verified

Add evaluation results for HLE, GPQA, AIME, HMMT, SWE-Bench, and Terminal-Bench (#4)

d9cb81b

add tech blog

36d7872

update footnote

e69e669
verified

Update README.md

20008c5
verified

Update README.md

4931c56
verified

Add files using upload-large-folder tool

e0ee936

initial commit

344a634
verified