Harrier OSS V1 27B - GGUF
This repository contains GGUF quantized formats of the massive 27-billion parameter microsoft/harrier-oss-v1-27b embedding model.
By utilizing these GGUF quants, you can run state-of-the-art semantic representation and text embedding generation on standard consumer hardware and CPUs.
Available Quantizations
| File Name | Bit Depth | Description |
|---|---|---|
| harrier-27b-Q8_0.gguf | 8-bit | Highest quality, virtually indistinguishable from FP16. Recommended if you have 32GB+ RAM. |
| harrier-27b-Q6_K.gguf | 6-bit | Excellent balance of quality and size. |
| harrier-27b-Q5_K_M.gguf | 5-bit | Great middle ground for memory constraints. |
| harrier-27b-Q4_K_M.gguf | 4-bit | Smallest footprint (~16.6 GB). Runs comfortably on machines with 24GB or 32GB of RAM. |
- Downloads last month
- 1,314
Hardware compatibility
Log In to add your hardware
4-bit
5-bit
6-bit
8-bit
Model tree for Abiray/harrier-oss-v1-27b-GGUF
Base model
microsoft/harrier-oss-v1-27b