Spaces:
Sleeping
Sleeping
Blog v3: Related Work + embedded dashboard + raw data links
Browse files
Blog.md
CHANGED
|
@@ -221,6 +221,30 @@ The environment is the contribution. The model is proof it works. Scaling is one
|
|
| 221 |
|
| 222 |
---
|
| 223 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 224 |
## Try It
|
| 225 |
|
| 226 |
Everything is open-source. Clone, install, run:
|
|
@@ -236,6 +260,10 @@ python inference.py --mode heuristic # No GPU needed
|
|
| 236 |
- 🤗 [Trained Model](https://huggingface.co/Timusgeorge/SynthAudit-Qwen2.5-3B-GRPO)
|
| 237 |
- 🔬 [Interactive Dashboard](https://huggingface.co/spaces/Timusgeorge/SynthAudit-Env)
|
| 238 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 239 |
```bibtex
|
| 240 |
@misc{saraswat2026synthaudit,
|
| 241 |
title={SynthAudit.Env: Multi-Agent Clinical AI Oversight via GRPO},
|
|
|
|
| 221 |
|
| 222 |
---
|
| 223 |
|
| 224 |
+
### Training Dashboard (4-Panel View)
|
| 225 |
+
|
| 226 |
+

|
| 227 |
+
|
| 228 |
+
---
|
| 229 |
+
|
| 230 |
+
## Why This Approach Is Different
|
| 231 |
+
|
| 232 |
+
There's growing work on AI safety in healthcare. Here's where SynthAudit.Env fits:
|
| 233 |
+
|
| 234 |
+
| Approach | What It Does | What It Misses |
|
| 235 |
+
|----------|-------------|----------------|
|
| 236 |
+
| **MedQA / USMLE benchmarks** | Tests medical knowledge | No adversarial reasoning, no multi-agent dynamics |
|
| 237 |
+
| **Red-teaming (manual)** | Humans find model failures | Doesn't scale, can't train an oversight agent |
|
| 238 |
+
| **Constitutional AI** | Self-critique via rules | No investigation tools, no raw data verification |
|
| 239 |
+
| **NurseSim-RL** (HF blog) | RL for clinical triage | Single-agent, no adversarial Actor |
|
| 240 |
+
| **SynthAudit.Env (ours)** | Multi-agent oversight with adversarial error injection, 8 investigation tools, Theory-of-Mind scoring, dense shaped rewards | — |
|
| 241 |
+
|
| 242 |
+
The key difference: we don't test whether a model *knows* medicine. We test whether a model can *catch another model* when it's confidently wrong. That's a fundamentally different capability — one that becomes critical as AI systems are deployed in clinical pipelines where the cost of undetected errors is measured in human lives.
|
| 243 |
+
|
| 244 |
+
No existing benchmark combines adversarial multi-agent dynamics, tool-augmented investigation, and RL-trainable oversight in a clinical domain.
|
| 245 |
+
|
| 246 |
+
---
|
| 247 |
+
|
| 248 |
## Try It
|
| 249 |
|
| 250 |
Everything is open-source. Clone, install, run:
|
|
|
|
| 260 |
- 🤗 [Trained Model](https://huggingface.co/Timusgeorge/SynthAudit-Qwen2.5-3B-GRPO)
|
| 261 |
- 🔬 [Interactive Dashboard](https://huggingface.co/spaces/Timusgeorge/SynthAudit-Env)
|
| 262 |
|
| 263 |
+
**Raw Data** (verify every claim):
|
| 264 |
+
- [`training_log_200.json`](https://huggingface.co/spaces/Timusgeorge/SynthAudit-Env/blob/main/outputs/training_log_200.json) — all 200 reward values
|
| 265 |
+
- [`post_training_eval.json`](https://huggingface.co/spaces/Timusgeorge/SynthAudit-Env/blob/main/outputs/post_training_eval.json) — base vs trained evaluation
|
| 266 |
+
|
| 267 |
```bibtex
|
| 268 |
@misc{saraswat2026synthaudit,
|
| 269 |
title={SynthAudit.Env: Multi-Agent Clinical AI Oversight via GRPO},
|