---
license: apache-2.0
library_name: mlx
base_model:
- Qwen/Qwen3.5-4B
- mlx-community/Qwen3.5-4B-MLX-4bit
pipeline_tag: text-generation
tags:
- mlx
- apple-silicon
- speculative-decoding
- qwen
- qwen3
- qwen3_5
- mtp
- mtplx
- local-ai
- q4
---

# Qwen3.5-4B MTPLX Optimized Speed (Q4 trunk)

## Run this with MTPLX

**MTPLX** is an MLX-native runtime for native Multi-Token-Prediction speculative decoding on Apple Silicon. Up to **2.24× faster decode** at real coding temperatures (`temp=0.6 / top_p=0.95 / top_k=20`) using the model's own built-in MTP heads — no external drafter, no greedy hack.

```bash
pip install mtplx
mtplx start
```

**Project:** [github.com/youssofal/MTPLX](https://github.com/youssofal/MTPLX)

**Other MTPLX checkpoints:**

- [Qwen3.6-27B-MTPLX-Optimized-Speed](https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed) — 4-bit flagship speed (63 TPS on M5 Max)
- [Qwen3.6-27B-MTPLX-Optimized](https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized) — verified default (GDN8-Speed4 trunk + CyanKiwi INT4 MTP)
- [Qwen3.5-4B-Optimized-MTPLX](https://huggingface.co/Youssofal/Qwen3.5-4B-Optimized-MTPLX) — small 8-bit

---

Small speed-test artifact for MTPLX on Apple Silicon.

This model uses the public `mlx-community/Qwen3.5-4B-MLX-4bit` MLX affine 4-bit
trunk and grafts back the official native MTP head from `Qwen/Qwen3.5-4B`. The
MTP head is stored as `mtp.safetensors`; layer-0 attention/MLP linears are
quantized to 4-bit affine group-64, while `mtp.fc` and the MTP norms stay BF16.

## Intended Use

A quick MTPLX download / load / speed-path test artifact at 4B scale. Once the
runtime ships:

```bash
mtplx start
```

Choose `Custom Hugging Face repo`, then enter:

```text
Youssofal/Qwen3.5-4B-MTPLX-Optimized-Speed
```

## Artifact Layout

- Trunk: MLX affine 4-bit, group size 64
- MTP sidecar: official Qwen3.5-4B MTP tensors
- MTP sidecar quantization: body-int4
- Runtime contract: `mtplx_runtime.json`
- MTPLX default: depth 2, target temperature 0.6, draft temperature 0.6

## Local Smoke Result

On the local Apple Silicon MTPLX workstation, the depth-2 speed path measured
**120.06 tok/s** versus **108.41 tok/s** AR on the warm-code prompt
(`max_tokens=48`, `temperature=0.6`, `top_p=0.95`, `top_k=20`). Depth 3 is
intentionally not the default for this 4B artifact because it over-drafts the
small native-MTP head.

## Build Stats

```json
{
  "bits": 4,
  "group_size": 64,
  "mode": "affine",
  "output_size_bytes": 86701040,
  "output_tensor_count": 29,
  "policy": "cyankiwi",
  "quantization": "body-int4",
  "quantized_linears": {
    "mtp.layers.0.mlp.down_proj":   {"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.mlp.gate_proj":   {"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.mlp.up_proj":     {"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.k_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.o_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.q_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.v_proj":{"bits": 4, "group_size": 64, "mode": "affine"}
  },
  "source_tensor_count": 15
}
```

## Links

- **MTPLX**: [github.com/youssofal/MTPLX](https://github.com/youssofal/MTPLX)  ·  `pip install mtplx`
- **Base model**: [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)
- **Trunk source**: [mlx-community/Qwen3.5-4B-MLX-4bit](https://huggingface.co/mlx-community/Qwen3.5-4B-MLX-4bit)