Upload folder using huggingface_hub

README.md

@@ -76,6 +76,14 @@ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
 print(response)
 ```
 
+## GGUF Quantized Versions
+
+Quantized GGUF versions are available in the `gguf/` directory for use with llama.cpp:
+
+- `RRT1-3B-q4_k_m.gguf` - 4-bit quantization (recommended for most use cases)
+- `RRT1-3B-q5_k_m.gguf` - 5-bit quantization (higher quality)
+- `RRT1-3B-q8_0.gguf` - 8-bit quantization (highest quality)
+
 ## Limitations
 
 May hallucinate or provide incorrect information. Not suitable for critical decision making.