---
base_model: unsloth/Qwen2.5-7B-Instruct
library_name: gguf
license: apache-2.0
tags: [gguf, tool-use, radicle, git, qwen2, qlora]
datasets: [h-d-h/rad-model-dataset]
---

# rad-model

QLoRA fine-tune of [Qwen 2.5 7B Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) for Radicle and Git tool calling. Distributed as a GGUF file for local inference with [llama.cpp](https://github.com/ggerganov/llama.cpp).

## Intended use

Tool-calling backend for CLI assistants that work with [Radicle](https://radicle.xyz) (decentralized code collaboration) and Git. The model selects and parameterizes the right CLI tool given a natural language request.

## Training

- **Method**: QLoRA (r=32, alpha=64) via [Unsloth](https://github.com/unslothai/unsloth)
- **Dataset**: [h-d-h/rad-model-dataset](https://huggingface.co/datasets/h-d-h/rad-model-dataset) — ~870 synthetic tool-calling examples covering 89 tools
- **Hardware**: NVIDIA RTX 3090 (24 GB VRAM)
- **Quantization**: Q4_K_M (~4.5 GB)

## Serving

```bash
llama-server -m rad-model-run6-q4_k_m.gguf --port 8080 -ngl 99 --host 0.0.0.0
```

The model serves an OpenAI-compatible `/v1/chat/completions` endpoint with tool-calling support.

## Evaluation

Evaluated on 88 held-out examples (stratified across all 89 tools). Scoring: 1.0 = correct tool + arguments, 0.75 = correct tool + extra args, 0.5 = correct tool + wrong args, 0.0 = wrong tool or no tool call.

See [RESULTS.md](https://app.radicle.xyz/nodes/rosa.radicle.xyz/rad:z2YCwgkXrZkUTu8c4CQayvk9Pkpky/tree/RESULTS.md) for full experiment history.

## Limitations

- Trained on synthetic data only; may not handle ambiguous real-world requests well
- Tool descriptions heavily influence accuracy — the base model with good descriptions can outperform the fine-tune on some tasks
- English only
- Designed for single-turn or short multi-turn tool-calling; not a general chat model

## Source

Developed on [Radicle](https://radicle.xyz): `rad:z2YCwgkXrZkUTu8c4CQayvk9Pkpky`

## License

Apache-2.0