Update README.md
Browse files
README.md
CHANGED
|
@@ -14,4 +14,4 @@ name: RedHatAI/Trinity-Large-Thinking-NVFP4
|
|
| 14 |
This is a preliminary version (and subject to change) of NVFP4 quantized [arcee-ai/Trinity-Large-Thinking ](https://huggingface.co/arcee-ai/Trinity-Large-Thinking/tree/main ) model.
|
| 15 |
The model has both weights and activations quantized to NVFP4 format with [vllm-project/llm-compressor](https://github.com/vllm-project/llm-compressor).
|
| 16 |
|
| 17 |
-
It is compatible and tested against vllm main.
|
|
|
|
| 14 |
This is a preliminary version (and subject to change) of NVFP4 quantized [arcee-ai/Trinity-Large-Thinking ](https://huggingface.co/arcee-ai/Trinity-Large-Thinking/tree/main ) model.
|
| 15 |
The model has both weights and activations quantized to NVFP4 format with [vllm-project/llm-compressor](https://github.com/vllm-project/llm-compressor).
|
| 16 |
|
| 17 |
+
It is compatible and tested against vllm main. Run it with ```vllm serve RedHatAI/Trinity-Large-Thinking-NVFP4 --trust-remote-code```
|