metadata
base_model: unsloth/Qwen2.5-7B-Instruct
library_name: gguf
license: apache-2.0
tags:
- gguf
- tool-use
- radicle
- git
- qwen2
- qlora
datasets:
- h-d-h/rad-model-dataset
rad-model
QLoRA fine-tune of Qwen 2.5 7B Instruct for Radicle and Git tool calling. Distributed as a GGUF file for local inference with llama.cpp.
Intended use
Tool-calling backend for CLI assistants that work with Radicle (decentralized code collaboration) and Git. The model selects and parameterizes the right CLI tool given a natural language request.
Training
- Method: QLoRA (r=32, alpha=64) via Unsloth
- Dataset: h-d-h/rad-model-dataset — ~870 synthetic tool-calling examples covering 89 tools
- Hardware: NVIDIA RTX 3090 (24 GB VRAM)
- Quantization: Q4_K_M (~4.5 GB)
Serving
llama-server -m rad-model-run6-q4_k_m.gguf --port 8080 -ngl 99 --host 0.0.0.0
The model serves an OpenAI-compatible /v1/chat/completions endpoint with tool-calling support.
Evaluation
Evaluated on 88 held-out examples (stratified across all 89 tools). Scoring: 1.0 = correct tool + arguments, 0.75 = correct tool + extra args, 0.5 = correct tool + wrong args, 0.0 = wrong tool or no tool call.
See RESULTS.md for full experiment history.
Limitations
- Trained on synthetic data only; may not handle ambiguous real-world requests well
- Tool descriptions heavily influence accuracy — the base model with good descriptions can outperform the fine-tune on some tasks
- English only
- Designed for single-turn or short multi-turn tool-calling; not a general chat model
Source
Developed on Radicle: rad:z2YCwgkXrZkUTu8c4CQayvk9Pkpky
License
Apache-2.0