File size: 2,330 Bytes
16145bd 7a2880f a3854e0 16145bd 56b0e55 16145bd 56b0e55 16145bd 56b0e55 16145bd 56b0e55 16145bd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
license: apache-2.0
base_model: Qwen/Qwen3-4B
tags:
- hallucination-detection
- token-classification
- qwen3
language:
- en
---
# TokenHD-4B
**TokenHD** is a token-level hallucination detector trained on top of [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) using the TokenHD pipeline. It assigns a hallucination probability to each token in an LLM-generated response, enabling fine-grained localization of errors without requiring predefined step segmentation.
Paper: [arxiv.org/abs/2605.12384](https://arxiv.org/abs/2605.12384)
Code: [github.com/rmin2000/TokenHD](https://github.com/rmin2000/TokenHD)
Training Data: [mr233/TokenHD-training-data](https://huggingface.co/datasets/mr233/TokenHD-training-data)
---
## Model Details
| Property | Value |
|---|---|
| Base model | `Qwen/Qwen3-4B` |
| Architecture | `AutoModelForTokenClassification` (`num_labels=1`) |
| Training domain | Mathematics (competition-level problems) |
| Output | Per-token hallucination probability (sigmoid of logits) |
---
## Usage
```python
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
model_id = "mr233/TokenHD-4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id, num_labels=1)
model.eval()
problem = "What is the capital of France?"
response = "The capital of France is London."
messages = [
{"role": "user", "content": problem},
{"role": "assistant", "content": response},
]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=False)[:-2]
input_tensor = torch.tensor(input_ids).unsqueeze(0)
with torch.no_grad():
logits = model(input_ids=input_tensor).logits # shape: (1, seq_len, 1)
# scores for response tokens only
response_ids = tokenizer.encode(response, add_special_tokens=False)
scores = torch.sigmoid(logits.squeeze(-1).squeeze(0))[-len(response_ids):]
# scores[i] is the hallucination probability for the i-th response token
```
---
## Evaluation
TokenHD models are evaluated with two metrics:
- **S_incor**: Token-level F1 on hallucinated (incorrect) responses — measures how precisely the detector localizes errors.
- **S_cor**: Recall on hallucination-free (correct) responses — measures how rarely the detector raises false alarms.
|