nvidia

/

llama-3.1-nemoguard-8b-content-safety

like 33

Follow

NVIDIA 55.4k

You can use their API:
https://build.nvidia.com/nvidia/llama-3_1-nemoguard-8b-content-safety

I want to run it locally. :)

Hi, did you able to run it locally? can you share sample code?

Hi, did you able to run it locally? can you share sample code?
Hi, check this guide: https://docs.nvidia.com/nim/llama-3-1-nemotron-safety-guard-8b/1.14.0/getting-started.html
If you are using vllm, you need to first download the adpator config, load base model with use lora enabled and the call the generate/chat method by passing this config directory.
The guide for using this model was unclear to me, after reading the above guide I find no system prompt is really needed for this model. Everything should be sent through the user prompt.

Hope it helps 😀