HuggingFaceBio/Carbon-3B-GGUF

Carbon-3B GGUF

GGUF (bf16) conversion of HuggingFaceBio/Carbon-3B for use with llama.cpp.

Carbon is a hybrid DNA / English language model that switches between Qwen3-4B-Base byte-level BPE for natural text and fixed 6-mer chunking for DNA inside <dna>...</dna> tags.

Requires a recent llama.cpp

HybridDNATokenizer support was merged in ggml-org/llama.cpp#23410, so any build from master after that works:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

Files

File	Quant	Size
carbon-3b-bf16.gguf	bf16 (lossless from source)	6.5 GB

Usage

Download

hf download HuggingFaceBio/Carbon-3B-GGUF carbon-3b-bf16.gguf --local-dir .

Basic DNA completion

./build/bin/llama-completion -m carbon-3b-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 64 --temp 0 -no-cnv

Metadata-conditioned generation

./build/bin/llama-completion -m carbon-3b-bf16.gguf \
    -p '<vertebrate_mammalian><protein_coding_region><dna>ATGCGCTAG' \
    -n 64 --temp 0 -no-cnv

Speculative decoding with Carbon-500M draft (~1.35x speedup)

Carbon-500M shares the HybridDNA vocab, so it works as a drop-in draft model. Grab it first:

hf download HuggingFaceBio/Carbon-500M-GGUF carbon-500m-bf16.gguf --local-dir .

Then run the standalone tool with --model-draft:

./build/bin/llama-speculative \
    -m  carbon-3b-bf16.gguf \
    -md carbon-500m-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 256 --temp 0

Or serve the 3B with the 500M draft (llama-server accepts the same -md flag):

./build/bin/llama-server \
    -m  carbon-3b-bf16.gguf \
    -md carbon-500m-bf16.gguf \
    --draft-max 16 --draft-min 1 \
    --port 8080

curl -s http://localhost:8080/completion -d '{
  "prompt": "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG",
  "n_predict": 256,
  "temperature": 0
}' | jq -r .content

Likelihood scoring

The source card's Python score() function computes mean log-prob per DNA token. In llama.cpp the closest tools are llama-perplexity for corpus-level perplexity (perplexity = exp(-mean_logprob)):

# one prompt per line in dna_corpus.txt, each wrapped in <dna>...</dna>
./build/bin/llama-perplexity -m carbon-3b-bf16.gguf -f dna_corpus.txt --ppl-stride 0

Or llama-server with logprobs for per-token log-probabilities:

./build/bin/llama-server -m carbon-3b-bf16.gguf --port 8080 &
curl -s http://localhost:8080/completion -d '{
  "prompt": "<dna>GGGCTATAAAGGCCATCGATCGATCGATCGATCGATCGATCG</dna>",
  "n_predict": 0,
  "n_probs": 1
}' | jq '.completion_probabilities'

Long context with YaRN (65k tokens ≈ 393 kbp)

Mirrors the source card's rope_scaling = {type: yarn, factor: 4.0, original_max_position_embeddings: 32768}:

./build/bin/llama-completion -m carbon-3b-bf16.gguf \
    -c 65536 --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768 \
    -p '<dna>...' -n 64 --temp 0 -no-cnv

Base-pair-level generation (FNS branch) — not supported

The revision="fns" example from the source card needs custom modeling code (factorized nucleotide supervision head), which only the Python transformers path can load. llama.cpp can't run that branch.

Tokenization parity

llama.cpp produces byte-for-byte identical token IDs to the Python HybridDNATokenizer (loaded with trust_remote_code=True) on the standard <dna>/metadata/edge-case fixtures shipped in ggml-org/vocabs.

See also

• Source weights: HuggingFaceBio/Carbon-3B
• Other GGUF variants: 500M · 3B · 8B

License

Apache-2.0, inherited from the source model.