🧬 Carbon
Collection
Carbon 500M, 3B, 8B genomic models and GGUF variants for llama.cpp • 6 items • Updated • 27
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf HuggingFaceBio/Carbon-500M-GGUF:BF16# Run inference directly in the terminal:
llama-cli -hf HuggingFaceBio/Carbon-500M-GGUF:BF16Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf HuggingFaceBio/Carbon-500M-GGUF:BF16# Run inference directly in the terminal:
./llama-cli -hf HuggingFaceBio/Carbon-500M-GGUF:BF16Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf HuggingFaceBio/Carbon-500M-GGUF:BF16# Run inference directly in the terminal:
./build/bin/llama-cli -hf HuggingFaceBio/Carbon-500M-GGUF:BF16Use Docker
docker model run hf.co/HuggingFaceBio/Carbon-500M-GGUF:BF16Quick Links
Carbon-500M GGUF
GGUF (bf16) conversion of HuggingFaceBio/Carbon-500M for use with llama.cpp.
Carbon is a hybrid DNA / English language model that switches between Qwen3-4B-Base byte-level BPE for natural text and fixed 6-mer chunking for DNA inside <dna>...</dna> tags.
Requires a recent llama.cpp
HybridDNATokenizer support was merged in ggml-org/llama.cpp#23410, so any build from master after that works:
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j
Files
| File | Quant | Size |
|---|---|---|
carbon-500m-bf16.gguf |
bf16 (lossless from source) | 983 MB |
Usage
Download
hf download HuggingFaceBio/Carbon-500M-GGUF carbon-500m-bf16.gguf --local-dir .
Basic DNA completion
./build/bin/llama-completion -m carbon-500m-bf16.gguf \
-p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
-n 64 --temp 0 -no-cnv
As a draft model for speculative decoding
Carbon-500M shares the HybridDNA vocab with the larger models, so it makes an excellent draft model:
# 8B target + 500M draft -> ~2x speedup at temp=0
./build/bin/llama-speculative \
-m carbon-8b-bf16.gguf \
-md carbon-500m-bf16.gguf \
-p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
-n 256 --temp 0
See also
- Source weights: HuggingFaceBio/Carbon-500M
- Other GGUF variants: 500M · 3B · 8B
License
Apache-2.0, inherited from the source model.
- Downloads last month
- -
Hardware compatibility
Log In to add your hardware
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for HuggingFaceBio/Carbon-500M-GGUF
Base model
HuggingFaceBio/Carbon-500M
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf HuggingFaceBio/Carbon-500M-GGUF:BF16# Run inference directly in the terminal: llama-cli -hf HuggingFaceBio/Carbon-500M-GGUF:BF16