latent-lab/sp-en-trans-truth-qwen2.5-1.5b

lmprobe: Linear Probe on Qwen2.5-1.5B

Truth probe for 'The Spanish word X means Y' statements. High accuracy (98.6%) — translation knowledge is linearly accessible at 1.5B scale.

Classes

• 0: false_statement
• 1: true_statement

Usage

from lmprobe import LinearProbe

probe = LinearProbe.from_hub("latent-lab/sp-en-trans-truth-qwen2.5-1.5b", trust_classifier=True)
predictions = probe.predict(["your text here"])

Probe Details

• Base model: Qwen/Qwen2.5-1.5B
• Model revision: 8faed761d45a263340a0528343f099c05c9a4323
• Layers: all (0–27, 28 layers)
• Pooling: last_token
• Classifier: logistic_regression
• Task: classification
• Random state: 42

Evaluation

Metric	Value
accuracy	0.9861
auroc	1.0000
f1	0.9859
precision	1.0000
recall	0.9722

Training Data

• Positive examples: 141

• Negative examples: 141

• Positive hash: sha256:58eb2ac584a1efd76a382f73b759449abe782ffcfc6889637ddb1f341d23ceab

• Negative hash: sha256:1808ffac12e8da0ef06f61afe46a2cc3d6b86cf913714ca2caef21262e3a925d

• Evaluation samples: 72

• Evaluation hash: sha256:20e4dec1b8225ce46aceae422df99fd84a9f7ee4cf32a54415aab9316163bcb6

Reproducibility

• lmprobe version: 0.5.8
• Python: 3.12.3
• PyTorch: 2.10.0+cu128
• scikit-learn: 1.8.0
• transformers: 5.3.0