Add model card
Browse files
README.md
CHANGED
|
@@ -1,3 +1,89 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
license: apache-2.0
|
| 5 |
+
base_model: distilbert/distilbert-base-uncased
|
| 6 |
+
tags:
|
| 7 |
+
- text-classification
|
| 8 |
+
- guardrail
|
| 9 |
+
- safety
|
| 10 |
+
- cbt
|
| 11 |
+
- mental-health
|
| 12 |
+
- academic-stress
|
| 13 |
+
pipeline_tag: text-classification
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# ReframeBot-Guardrail-DistilBERT
|
| 17 |
+
|
| 18 |
+
A 3-class text classifier fine-tuned from `distilbert-base-uncased` that
|
| 19 |
+
routes conversation turns to one of three task modes:
|
| 20 |
+
|
| 21 |
+
| Label | Meaning |
|
| 22 |
+
|---|---|
|
| 23 |
+
| `TASK_1` | CBT / academic stress — engage with Socratic questioning |
|
| 24 |
+
| `TASK_2` | Crisis / self-harm signal — redirect to emergency hotlines |
|
| 25 |
+
| `TASK_3` | Out-of-scope — validate feeling, pivot back to academics |
|
| 26 |
+
|
| 27 |
+
This model is the guardrail component of the ReframeBot system. Its output
|
| 28 |
+
is combined with a dual-signal crisis detector (regex + cosine similarity)
|
| 29 |
+
before the final routing decision is made.
|
| 30 |
+
|
| 31 |
+
## Usage
|
| 32 |
+
|
| 33 |
+
```python
|
| 34 |
+
from transformers import pipeline
|
| 35 |
+
|
| 36 |
+
classifier = pipeline(
|
| 37 |
+
"text-classification",
|
| 38 |
+
model="Nhatminh1234/ReframeBot-Guardrail-DistilBERT",
|
| 39 |
+
)
|
| 40 |
+
|
| 41 |
+
classifier("I'm really stressed about my finals next week")
|
| 42 |
+
# [{'label': 'TASK_1', 'score': 0.97}]
|
| 43 |
+
|
| 44 |
+
classifier("What's a good recipe for pasta?")
|
| 45 |
+
# [{'label': 'TASK_3', 'score': 0.94}]
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
## Training Details
|
| 49 |
+
|
| 50 |
+
| Hyperparameter | Value |
|
| 51 |
+
|---|---|
|
| 52 |
+
| Base model | distilbert-base-uncased |
|
| 53 |
+
| Number of labels | 3 (TASK_1, TASK_2, TASK_3) |
|
| 54 |
+
| Learning rate | 2e-6 |
|
| 55 |
+
| Batch size | 16 |
|
| 56 |
+
| Max epochs | 20 (early stopping, patience=3) |
|
| 57 |
+
| Weight decay | 0.01 |
|
| 58 |
+
| Max token length | 128 |
|
| 59 |
+
| Best model criterion | macro F1 |
|
| 60 |
+
| Hardware | NVIDIA RTX 5070 (laptop, 8 GB VRAM) |
|
| 61 |
+
|
| 62 |
+
**Dataset:** 1,674 labelled samples (80/20 train/val split). Includes hard
|
| 63 |
+
negatives — benign metaphors that superficially resemble crisis language
|
| 64 |
+
(e.g., "dying of embarrassment after that presentation").
|
| 65 |
+
|
| 66 |
+
## Evaluation
|
| 67 |
+
|
| 68 |
+
Per-class results on the validation split (335 samples):
|
| 69 |
+
|
| 70 |
+
| Class | Precision | Recall | F1 | Support |
|
| 71 |
+
|---|---|---|---|---|
|
| 72 |
+
| TASK_1 | 0.99 | 1.00 | 1.00 | 107 |
|
| 73 |
+
| TASK_2 | 0.98 | 1.00 | 0.99 | 91 |
|
| 74 |
+
| TASK_3 | 1.00 | 0.98 | 0.99 | 137 |
|
| 75 |
+
| **macro avg** | **0.99** | **0.99** | **0.99** | **335** |
|
| 76 |
+
|
| 77 |
+
Accuracy on a separate, harder held-out test set: **91.1%** (includes
|
| 78 |
+
boundary cases not present in the training distribution).
|
| 79 |
+
|
| 80 |
+
## Intended Use
|
| 81 |
+
|
| 82 |
+
Designed as a routing component in the ReframeBot system. The TASK_2
|
| 83 |
+
output alone is not sufficient for crisis intervention — the full system
|
| 84 |
+
also applies a regex + semantic similarity layer before acting on a
|
| 85 |
+
crisis signal.
|
| 86 |
+
|
| 87 |
+
## Project
|
| 88 |
+
|
| 89 |
+
GitHub: [ReframeBot](https://github.com/minhnghiem32131024429/ReframeBot)
|