Instructions to use nvidia/Nemotron-Labs-Diffusion-8B-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/Nemotron-Labs-Diffusion-8B-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/Nemotron-Labs-Diffusion-8B-Base", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("nvidia/Nemotron-Labs-Diffusion-8B-Base", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use nvidia/Nemotron-Labs-Diffusion-8B-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/Nemotron-Labs-Diffusion-8B-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Nemotron-Labs-Diffusion-8B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/Nemotron-Labs-Diffusion-8B-Base

SGLang

How to use nvidia/Nemotron-Labs-Diffusion-8B-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/Nemotron-Labs-Diffusion-8B-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Nemotron-Labs-Diffusion-8B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/Nemotron-Labs-Diffusion-8B-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Nemotron-Labs-Diffusion-8B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nvidia/Nemotron-Labs-Diffusion-8B-Base with Docker Model Runner:
```
docker model run hf.co/nvidia/Nemotron-Labs-Diffusion-8B-Base
```

Nemotron-Labs-Diffusion-8B-Base

Commit History

Clean up rope params; ensure transformers 4.55/5.0 compatibility

a4574ae
verified

abhgarg commited on 7 days ago

Upload tokenizer

85eb9d3
verified

YongganFu commited on 19 days ago

Upload model

f318bfe
verified

YongganFu commited on 19 days ago

Trying new settings for tokenizer_config.json to hopefully fix issues

d42bc62

trias702 commited on Feb 5

Update chat_template.jinja

0a9534a
verified

YongganFu commited on Feb 2

Changed chat_template to remove alternating check

0820ac4

trias702 commited on Jan 30

Update chat_template.jinja

abf48b3
verified

YongganFu commited on Jan 30

Update config.json

8200ec0
verified

YongganFu commited on Jan 28

Upload model

15597b8
verified

YongganFu commited on Jan 28

Upload model

f9e0c41
verified

YongganFu commited on Jan 28

set default causal_context=True

262d402

trias702 commited on Jan 22

Overriding the old function doesn't work, reverting to old approach

33b2954

trias702 commited on Jan 20

Trying to force transformers to use the older causal mask

456e96b

trias702 commited on Jan 20

Made some potential fixes for DSA, need to test if they work

ff07748

trias702 commited on Jan 20

Upload model

4cabc4d
verified

YongganFu commited on Jan 18

Upload model

6294f2a
verified

YongganFu commited on Jan 18

Removed p_mask assert for compatibility with nemo-rl

bad4dec

trias702 commited on Jan 18

Delete model.safetensors.index.json

b4fbb98
verified

YongganFu commited on Jan 18

Delete model-00002-of-00004.safetensors

987c652
verified

YongganFu commited on Jan 18

Delete model-00001-of-00004.safetensors

57e8939
verified

YongganFu commited on Jan 18

Delete model-00003-of-00004.safetensors

c4c4881
verified

YongganFu commited on Jan 18

Delete model-00004-of-00004.safetensors

e93dcde
verified

YongganFu commited on Jan 18

Upload model

378400a
verified

YongganFu commited on Jan 18

Upload tokenizer

1c298a2
verified

YongganFu commited on Jan 18

Upload model

dfcb979
verified

YongganFu commited on Jan 18

Added custom MinistralDiffOutputWithPast return type and skip_loss functionality

418d9e4

trias702 commited on Jan 16

Update README.md

618a1c8
verified

YongganFu commited on Jan 16

Upload tokenizer

46c0f95
verified

YongganFu commited on Jan 16

Upload model

a195040
verified

YongganFu commited on Jan 16

initial commit

3fa953f
verified

pmolchanov commited on Jan 14

Commit History

Clean up rope params; ensure transformers 4.55/5.0 compatibility a4574ae verified

Upload tokenizer 85eb9d3 verified

Upload model f318bfe verified

Trying new settings for tokenizer_config.json to hopefully fix issues d42bc62

Update chat_template.jinja 0a9534a verified

Changed chat_template to remove alternating check 0820ac4

Update chat_template.jinja abf48b3 verified

Update config.json 8200ec0 verified

Upload model 15597b8 verified

Upload model f9e0c41 verified

set default causal_context=True 262d402

Overriding the old function doesn't work, reverting to old approach 33b2954

Trying to force transformers to use the older causal mask 456e96b

Made some potential fixes for DSA, need to test if they work ff07748

Upload model 4cabc4d verified

Upload model 6294f2a verified

Removed p_mask assert for compatibility with nemo-rl bad4dec

Delete model.safetensors.index.json b4fbb98 verified

Delete model-00002-of-00004.safetensors 987c652 verified

Delete model-00001-of-00004.safetensors 57e8939 verified

Delete model-00003-of-00004.safetensors c4c4881 verified

Delete model-00004-of-00004.safetensors e93dcde verified

Upload model 378400a verified

Upload tokenizer 1c298a2 verified

Upload model dfcb979 verified

Added custom MinistralDiffOutputWithPast return type and skip_loss functionality 418d9e4

Update README.md 618a1c8 verified

Upload tokenizer 46c0f95 verified

Upload model a195040 verified

initial commit 3fa953f verified

Clean up rope params; ensure transformers 4.55/5.0 compatibility

a4574ae
verified

Upload tokenizer

85eb9d3
verified

Upload model

f318bfe
verified

Trying new settings for tokenizer_config.json to hopefully fix issues

d42bc62

Update chat_template.jinja

0a9534a
verified

Changed chat_template to remove alternating check

0820ac4

Update chat_template.jinja

abf48b3
verified

Update config.json

8200ec0
verified

Upload model

15597b8
verified

Upload model

f9e0c41
verified

set default causal_context=True

262d402

Overriding the old function doesn't work, reverting to old approach

33b2954

Trying to force transformers to use the older causal mask

456e96b

Made some potential fixes for DSA, need to test if they work

ff07748

Upload model

4cabc4d
verified

Upload model

6294f2a
verified

Removed p_mask assert for compatibility with nemo-rl

bad4dec

Delete model.safetensors.index.json

b4fbb98
verified

Delete model-00002-of-00004.safetensors

987c652
verified

Delete model-00001-of-00004.safetensors

57e8939
verified

Delete model-00003-of-00004.safetensors

c4c4881
verified

Delete model-00004-of-00004.safetensors

e93dcde
verified

Upload model

378400a
verified

Upload tokenizer

1c298a2
verified

Upload model

dfcb979
verified

Added custom MinistralDiffOutputWithPast return type and skip_loss functionality

418d9e4

Update README.md

618a1c8
verified

Upload tokenizer

46c0f95
verified

Upload model

a195040
verified

initial commit

3fa953f
verified