Instructions to use Intel/DeepSeek-V4-Flash-W4A16-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Intel/DeepSeek-V4-Flash-W4A16-AutoRound with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Intel/DeepSeek-V4-Flash-W4A16-AutoRound")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Intel/DeepSeek-V4-Flash-W4A16-AutoRound")
model = AutoModelForCausalLM.from_pretrained("Intel/DeepSeek-V4-Flash-W4A16-AutoRound")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Intel/DeepSeek-V4-Flash-W4A16-AutoRound with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Intel/DeepSeek-V4-Flash-W4A16-AutoRound"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Intel/DeepSeek-V4-Flash-W4A16-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Intel/DeepSeek-V4-Flash-W4A16-AutoRound

SGLang

How to use Intel/DeepSeek-V4-Flash-W4A16-AutoRound with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Intel/DeepSeek-V4-Flash-W4A16-AutoRound" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Intel/DeepSeek-V4-Flash-W4A16-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Intel/DeepSeek-V4-Flash-W4A16-AutoRound" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Intel/DeepSeek-V4-Flash-W4A16-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Intel/DeepSeek-V4-Flash-W4A16-AutoRound with Docker Model Runner:
```
docker model run hf.co/Intel/DeepSeek-V4-Flash-W4A16-AutoRound
```

xinhe commited on about 17 hours ago

Commit

d8ac6c0

verified ·

1 Parent(s): a729b70

Upload folder using huggingface_hub

Browse files

Files changed (44) hide show

model-00003-of-00046.safetensors +3 -0
model-00004-of-00046.safetensors +3 -0
model-00005-of-00046.safetensors +3 -0
model-00006-of-00046.safetensors +3 -0
model-00007-of-00046.safetensors +3 -0
model-00008-of-00046.safetensors +3 -0
model-00009-of-00046.safetensors +3 -0
model-00010-of-00046.safetensors +3 -0
model-00011-of-00046.safetensors +3 -0
model-00012-of-00046.safetensors +3 -0
model-00013-of-00046.safetensors +3 -0
model-00014-of-00046.safetensors +3 -0
model-00015-of-00046.safetensors +3 -0
model-00016-of-00046.safetensors +3 -0
model-00017-of-00046.safetensors +3 -0
model-00018-of-00046.safetensors +3 -0
model-00019-of-00046.safetensors +3 -0
model-00020-of-00046.safetensors +3 -0
model-00021-of-00046.safetensors +3 -0
model-00022-of-00046.safetensors +3 -0
model-00023-of-00046.safetensors +3 -0
model-00024-of-00046.safetensors +3 -0
model-00025-of-00046.safetensors +3 -0
model-00026-of-00046.safetensors +3 -0
model-00027-of-00046.safetensors +3 -0
model-00028-of-00046.safetensors +3 -0
model-00029-of-00046.safetensors +3 -0
model-00030-of-00046.safetensors +3 -0
model-00031-of-00046.safetensors +3 -0
model-00032-of-00046.safetensors +3 -0
model-00033-of-00046.safetensors +3 -0
model-00034-of-00046.safetensors +3 -0
model-00035-of-00046.safetensors +3 -0
model-00036-of-00046.safetensors +3 -0
model-00037-of-00046.safetensors +3 -0
model-00038-of-00046.safetensors +3 -0
model-00039-of-00046.safetensors +3 -0
model-00040-of-00046.safetensors +3 -0
model-00041-of-00046.safetensors +3 -0
model-00042-of-00046.safetensors +3 -0
model-00043-of-00046.safetensors +3 -0
model-00044-of-00046.safetensors +3 -0
model-00045-of-00046.safetensors +3 -0
model-00046-of-00046.safetensors +3 -0

model-00003-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f7b1bd86d868e69226fcb113a3b6f71dc5caada7861459cc65a802727a1cd8cc
+size 3427419616

model-00004-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a797c4f0b787c68ff149f2d81f9ce73debd5d964915119dd0ff3cd0c4241df64
+size 3437385848

model-00005-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f5e88868fc91f138ceaa83da68066b7b622fe1c51a231776d2559b37dcec842c
+size 3423658256

model-00006-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1ddfe6f1d4bb4bebecf4aedd614bb126f07d004ddbdc8826f0a67db2c792d7a9
+size 3431181376

model-00007-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:28a6375b1d220b3eb3a375634eff3ab84b16abc5f4c08cd4dc0c3199b3d7cbed
+size 3423658256

model-00008-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0d8d1ac243d5a509d394a425d6784a021a9b3574d83cb9eaeb940857993f4e70
+size 3431181376

model-00009-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9d09ee1d9b392a33f720fe22e413fa50b18bcc5e9e711d97c52af257efc00b89
+size 3423658256

model-00010-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0a8d867d31d0fcd49b16824a9c28644b0bea07d4268ec727915ca389165d041f
+size 3431181376

model-00011-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:277248da439d1c4ca1eab81e0a189521415b3102e27018626f32656efc4691e4
+size 3423658256

model-00012-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e290660f0daf12abd56fe037ec4fcecfea71d16b3c5f14d570e857e79812b0de
+size 3431183736

model-00013-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cac0880c2dd03be152e9be5434874709b30b7ffc994a046ce8d59f47b128a770
+size 3423660608

model-00014-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3a938bd97f3b19b18d96646a56022d02cfcfa1a5812f6e1d16fae299355c60bd
+size 3431183736

model-00015-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fc8e4843ac478bf53faffd22f4087d18d7e1f5c5bdbf7b8ccf0ba2a997721ed7
+size 3423660608

model-00016-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d8ef297a833beb89de788024d24ba44de660e8fccb15ac0086356c54f0bf2a38
+size 3431183736

model-00017-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:837f53cdf4a22acca4aa23caec95bd1eed44ce9cdddb7be71bb4918170322e07
+size 3423660608

model-00018-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d2611d2e5274408335d2ceb2caa73c3715682fc7cbb211d9a31e6679410cfab2
+size 3431183736

model-00019-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e8fef4da43c21c9246aa0ee58417cf3c7915e64a49b93757cd8622aba60d0cd9
+size 3423660608

model-00020-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2e99e9b30c6551d26b50b0c316bdf270ea6599c493ba69187f4db8f6a0d1674f
+size 3431183736

model-00021-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:aa0ce177331636f80c30510b32625a2968a8cdb9817ab553a667ee484b55b0de
+size 3423660608

model-00022-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1beaa5dad0904caf68bd6779e6ea9feb4ac7606b9304bbd46f7fce344a9fe1bf
+size 3431183736

model-00023-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:46d6a3cd9e9621d56b679a80ac0ec16810fadce36ac460c2b8954204ed79c840
+size 3423660608

model-00024-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:85c3a76c7572800bf9a0e18284d9dbfdf309ce74256d51a77406d19bd2889620
+size 3431183736

model-00025-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0ef227c55d8d63126dcf3971472f48d49a5761b533105153be72a8764fda858d
+size 3423660608

model-00026-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f4ab1aff714b9a04da874ae3ca9f111c673a05a9ed98eb9afbf532900019a372
+size 3431183736

model-00027-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d100dc2fd6a40477e3cd2c1acf2563a62e19bbc8236229151de3cc8afb6c35e8
+size 3423660608

model-00028-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:585f545aa1a0fd2ba90b0bf73c4d4e782d021b5409f8732aaf672626e8b880e0
+size 3431183736

model-00029-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e881fd2800c332d92de455b5e66870448bd89875fbe253349e051c2092bee67a
+size 3423660608

model-00030-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:da88e3df8c32036b00adb3870df1c1563ec24fbc3587b2b409385bd98435be53
+size 3431183736

model-00031-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e2b2f58bd586eeb14105211900148722d57bcd98b7dec6e411a2742b0c1fb774
+size 3423660608

model-00032-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:66f14267c11cbe1fab8acf06a1bbe00fb9ffdd5647ee44e114b21f6799adb54d
+size 3431183736

model-00033-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b19498d784401b1d84ef1391a2b3ac96df34272ada471c96292e61258f326bc3
+size 3423660608

model-00034-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1c543d6e20eaa4aace7200221ebe58580b44db25b70cb8891c57cbe822309008
+size 3431183736

model-00035-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:96cdf8718ca753c080ccd425f4e73c90cf6af7500502f3b9197f36225c9d4949
+size 3423660608

model-00036-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2cdae8b3fea57596b0b1e75d7e25e988c96163ce6b6877d989feec9611a16a76
+size 3431183736

model-00037-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a8532307c61e5b4baa57d3a49c50cb9cabc8a39b3477ecf4de0eb2523d1cb01f
+size 3423660608

model-00038-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6aaaa2e5372f786618e3265802bc03d23a66dafed7834d7f5a39ddff7dbaa09f
+size 3431183736

model-00039-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5d539be25733a9f174ce380ef2120a8392c5c469ea77d2faf980e6079dc18e1c
+size 3423660608

model-00040-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0e4939f98e60a79c1e8c0d6ef861191925a55c2ee128e974ca1bf740f80612ff
+size 3431183736

model-00041-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:11668f1dc99abd3bbf274599d373dc839bd96df7e0113290e7e4b7590fad7466
+size 3423660608

model-00042-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6ddaae1f547ad5dfa7e595862c9513f19f88c9056de76ec74343fb34ffe7e4fc
+size 3431183736

model-00043-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0ceffbe4e3437d0b7eb5e42ca3feed75816f704aa7b32637d2e3da9bda24a0b2
+size 3423660608

model-00044-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:904f05584b895b34f70ba25f8a22b59fa5ee3a845253262016e91d3a7f5f468e
+size 3431183736

model-00045-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9a0fd242134e9ebe4e6993a7631692944838e4fdf20067b3219caa48eab68045
+size 1059332516

model-00046-of-00046.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:13ab4077fbf65b726535aec1cafce96758dd44bea0513126e31a22ecfd770b16
+size 3438928548