Upload README.md

README.md

@@ -0,0 +1,77 @@
+# GrandgemMa — Gemma 4 Scam Detection Eval & Fine-Tune Kit
+
+> **Goal:** Zero-shot test `google/gemma-4-E2B-it` (2B text) on real scam-call transcripts.  
+> If accuracy < 90 % or F1(SCAM) < 85 % → fine-tune with Unsloth 4-bit LoRA.
+
+## Quick Links
+
+| Artifact | Path |
+|---|---|
+| Zero-shot eval script | `eval_zero_shot.py` |
+| Unsloth SFT trainer | `train_sft_unsloth.py` |
+| Dataset formatter | `format_dataset.py` |
+| Decision rubric | below |
+
+## Datasets Used
+
+- **Primary:** [`BothBosu/scam-dialogue`](https://huggingface.co/datasets/BothBosu/scam-dialogue) — 800+ synthetic scam/legit call transcripts, labeled `dialogue` + `label` (1=SCAM, 0=LEGIT).
+- **Secondary (optional):** [`BothBosu/Scammer-Conversation`](https://huggingface.co/datasets/BothBosu/Scammer-Conversation) — extra mixed conversations.
+
+## 1. Zero-Shot Evaluation
+
+```bash
+# Quick smoke test (100 rows)
+python eval_zero_shot.py --limit 100
+
+# Full test split (~400 rows)
+python eval_zero_shot.py --limit -1
+
+# CPU-only fallback
+python eval_zero_shot.py --device cpu --dtype fp32 --limit 20
+```
+
+**Output:** `results_zero_shot.json` + console report with accuracy / precision / recall / F1 / confusion matrix.
+
+## 2. Decision Rubric
+
+| Condition | Verdict | Action |
+|---|---|---|
+| Accuracy ≥ 90 % **and** F1(SCAM) ≥ 85 % | ✅ PASS | Base model is strong enough. Fine-tuning optional. |
+| Accuracy 75–89 % **or** F1(SCAM) 70–84 % | ⚠️ MARGINAL | **Fine-tune recommended.** Expected uplift +5–15 pp. |
+| Accuracy < 75 % **or** F1(SCAM) < 70 % | ❌ FAIL | **Fine-tune REQUIRED.** Run `train_sft_unsloth.py`. |
+
+> **Why these thresholds?** For elder-scam defense, missing a scam (false negative) is catastrophic. High recall on SCAM class is mandatory.
+
+## 3. Fine-Tuning (Unsloth SFT)
+
+```bash
+# Install deps
+pip install unsloth transformers datasets trl peft accelerate
+
+# Train & push to HF Hub
+python train_sft_unsloth.py \
+  --output grandgemma-scam-sft \
+  --push_to_hub s23deepak/grandgemma-scam-sft
+```
+
+**Hardware:** Kaggle T4×2 (free) or any single GPU ≥ 16 GB VRAM.  
+**Config:** 4-bit + LoRA r=16, 3 epochs, lr=2e-4, batch=2, grad_accum=4.  
+**Expected time:** ~3–5 min / epoch on T4×2.
+
+## 4. Re-Eval After Fine-Tuning
+
+After training, run the same `eval_zero_shot.py` but point `--model` at your fine-tuned checkpoint:
+
+```bash
+python eval_zero_shot.py \
+  --model s23deepak/grandgemma-scam-sft \
+  --limit -1
+```
+
+Compare the delta in `accuracy`, `recall_scam`, and `f1_scam`.
+
+## 5. Team Notes
+
+- This repo is **evaluation-only** — no app code. App code lives in your monorepo (`/android`, `/ios`, `/extensions`, `/portal`).
+- Fine-tuned weights produced here should be quantized to `.litertlm` for on-device Android inference (Stream A) and converted for iOS/browser WebGPU (Stream B).
+- Track all runs in a spreadsheet: run_id | model | dataset | accuracy | f1_scam | notes.