Hugging Face's logo

henrik3
/
sweep-next-edit-v2-7B-NVFP4

Text Generation
Safetensors
English
qwen2
code
autocomplete
next-edit-prediction
dpo
8-bit precision
compressed-tensors
Model card Files
xet
 Community

Converted via llm compressor, with 200 examples from henrik3/sweep-calibration for calibration (which is to be fair not best quality but should be fine)

Performance

Sglang & RTX PRO 4500 (800GB/s bandwidth, Blackwell)

Command:

python3 -m sglang.launch_server --model-path henrik3/sweep-next-edit-v2-7B-NVFP4 --port 8000 --host 0.0.0.0 --trust-remote-code --mem-fraction-static 0.8 --context-length 16384 --speculative-algorithm NGRAM --speculative-num-draft-tokens 4

Stats

  • ~9100 input tokens
  • ~200 output tokens
  • ~300ms response time

Find the original model here: https://huggingface.co/sweepai/sweep-next-edit-v2-7B

Downloads last month
1,175
Safetensors
Model size
5B params
Tensor type
BF16
·
F32
·
F8_E4M3
·
U8
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for henrik3/sweep-next-edit-v2-7B-NVFP4

Base model

Qwen/Qwen2.5-7B
Finetuned
Qwen/Qwen2.5-Coder-7B
Finetuned
sweepai/sweep-next-edit-v2-7B
Quantized
(4)
this model

Dataset used to train henrik3/sweep-next-edit-v2-7B-NVFP4

Collection including henrik3/sweep-next-edit-v2-7B-NVFP4