Update README.md

README.md

@@ -12,66 +12,323 @@ tags:
   - lora
   - peft
   - nigeria
-  - naija
   - privacy-filter
 model-index:
-  - name: iamSamurai/privacy-filter-nigeria
+  - name: privacy-filter-nigeria
     results:
       - task:
           type: token-classification
           name: PII Span Detection
         dataset:
-          name: Naija privacy-filter training release
+          name: stage2_v5 private mixed dataset (validation)
           type: custom
         metrics:
           - type: f1
-            name: Validation typed span F1
-            value: 0.9737
+            name: Typed Span F1
+            value: 0.9763
+          - type: precision
+            name: Typed Span Precision
+            value: 0.9707
+          - type: recall
+            name: Typed Span Recall
+            value: 0.9820
+      - task:
+          type: token-classification
+          name: PII Span Detection
+        dataset:
+          name: stage2_v5 private mixed dataset (test)
+          type: custom
+        metrics:
+          - type: f1
+            name: Typed Span F1
+            value: 0.9640
+          - type: precision
+            name: Typed Span Precision
+            value: 0.9593
+          - type: recall
+            name: Typed Span Recall
+            value: 0.9688
+      - task:
+          type: token-classification
+          name: Hard-Negative False Positive Audit
+        dataset:
+          name: stage2_v5 hard-negative challenge
+          type: custom
+        metrics:
+          - type: false_positive_rate
+            name: False-positive example rate
+            value: 0.72
 ---
 
-# OpenAI Privacy Filter Naija LoRA
+# Privacy Filter Nigeria LoRA
 
-This is a PEFT LoRA adapter for `openai/privacy-filter` trained for Nigerian-domain PII
-span detection. It is a research preview, not a production privacy or
-compliance control.
+A LoRA adapter on top of [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter)
+for Nigerian-domain PII span detection. This is a **v0.1 research preview**,
+not a production privacy product.
 
-## Checkpoint
-
-- Adapter repo: `iamSamurai/privacy-filter-nigeria`
-- Base model: `openai/privacy-filter`
-- Best epoch: `9`
-- Best validation typed span F1: `0.9737`
+- **Adapter repo:** `iamSamurai/privacy-filter-nigeria`
+- **Base model:** `openai/privacy-filter`
+- **Adapter type:** LoRA (PEFT) for token classification
+- **Source repo:** https://github.com/iamNarcisse/naija-privacy-filter
+- **Eval artifacts:** [`iamSamurai/openai-privacy-filter-naija-eval-artifacts`](https://huggingface.co/iamSamurai/openai-privacy-filter-naija-eval-artifacts)
+- **License:** Apache-2.0 (this adapter). Use of the base model remains
+  subject to the upstream model's terms.
 
 ## Evaluation
 
-| Split | Typed span F1 |
+Latest v5 eval against the internal stage2 v5 private mixed dataset, after
+deterministic span postprocessing:
+
+| Split | Typed span F1 | Precision | Recall | TP | FP | FN |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| Validation | 0.9763 | 0.9707 | 0.9820 | 762 | 23 | 14 |
+| Test | 0.9640 | 0.9593 | 0.9688 | 777 | 33 | 25 |
+
+The v5 challenge split is hard-negative-only. Typed F1 is not meaningful
+because there are no gold positive spans. Use false-positive diagnostics:
+
+| Challenge diagnostic | Value |
 | --- | ---: |
-| Validation | 0.9737 |
-| Test | 0.9640 |
-| Challenge | 0.0000 |
+| Examples | 250 |
+| Examples with predictions | 180 |
+| False-positive example rate | 0.72 |
+| Predicted false-positive spans | 456 |
 
-The challenge split is diagnostic and should not be used for checkpoint
-selection. Review the included metric JSON files for full precision, recall,
-loss, and false-positive details.
+This is a recall-oriented research adapter. The hard-negative audit shows that
+benign identifier-like text can be over-redacted. Precision-sensitive users
+should add deterministic filters, tune thresholds where applicable, or
+finetune on representative local negatives.
 
-## Usage
+## How To Use
 
-Use the project runner so the classification head is resized from
-`label_map.json` before the adapter is loaded:
+> **Note on the classifier head.** This adapter ships a resized token-
+> classification head for the Nigerian-domain label taxonomy
+> (`label_map.json` / `token_label_names` in `adapter_config.json`). Loading
+> the adapter on top of the unmodified base model with vanilla
+> `peft.PeftModel.from_pretrained` will not resize the head automatically.
+> Use the project runner (`privacy_filter.py`) or replicate its head-resize
+> logic to get correct predictions.
+
+### Recommended: project runner
 
 ```bash
+pip install "torch>=2.8" "transformers>=4.56" "peft>=0.17" "huggingface-hub>=0.34"
 git clone https://github.com/iamNarcisse/naija-privacy-filter
 cd naija-privacy-filter
 
+python main.py \
+  --model-name openai/privacy-filter \
+  --adapter-name iamSamurai/privacy-filter-nigeria \
+  "Amina Yusuf can be reached at +234 802 111 3344."
+```
+
+Or via `uv`:
+
+```bash
 uv run python main.py \
   --model-name openai/privacy-filter \
   --adapter-name iamSamurai/privacy-filter-nigeria \
   "Amina Yusuf can be reached at +234 802 111 3344."
 ```
 
-## Limitations
+### REST API
+
+```bash
+PRIVACY_FILTER_MODEL_NAME=openai/privacy-filter \
+PRIVACY_FILTER_ADAPTER_NAME=iamSamurai/privacy-filter-nigeria \
+uv run uvicorn api:app --reload
+
+curl -X POST http://127.0.0.1:8000/predict \
+  -H "Content-Type: application/json" \
+  -d '{"text":"Amina Yusuf can be reached at +234 802 111 3344.","mode":"cleaned"}'
+```
+
+### Direct `transformers + peft` (advanced)
+
+If you want to bypass the project runner, you must resize the base model's
+classification head to match `token_label_names` from the adapter's
+`adapter_config.json` before applying the LoRA weights. See
+[`privacy_filter.py`](https://github.com/iamNarcisse/naija-privacy-filter/blob/main/privacy_filter.py)
+for the reference implementation.
+
+## Intended Use
+
+Use this adapter for:
+
+- Research and evaluation of Nigerian-domain PII detection.
+- Prototyping local inference or REST API integration for privacy-filtering
+  workflows.
+- Studying LoRA adaptation and deterministic span postprocessing for
+  token-classification models.
+- Producing candidate spans for downstream review, redaction, or policy
+  engines.
+
+Do **not** use this adapter as the only control for regulatory, legal,
+medical, financial, or irreversible privacy decisions.
+
+## Training Data
+
+The source repository includes a tiny public synthetic example bundle,
+`data/examples`:
+
+| Split | Examples |
+| --- | ---: |
+| Train | 5 |
+| Validation | 5 |
+| Test | 5 |
+| Challenge | 5 |
+
+This example bundle is for schema inspection and smoke tests only. It is not a
+training or evaluation release.
+
+The current v5 adapter was trained and evaluated against a later private
+stage2 v5 mixed dataset that is not distributed with this model card. That
+private mix includes synthetic examples, OCR-derived Nigerian
+identity-document samples used to test document-layout and OCR behavior, and
+real-world domain samples. Direct identifiers and sensitive fields were
+annotated and redacted from model-use fields. Source materials and derived
+artifacts remain private and are not distributed.
+
+Label space includes Nigerian-domain PII types such as `private_nin`,
+`private_bvn`, `account_number`, `private_passport_number`,
+`private_voters_card_number`, and `private_drivers_license_number`,
+alongside generic PII labels: `private_person`, `private_email`,
+`private_phone`, `private_address`, `private_date`, `private_url`, and
+`secret`.
+
+The committed public examples are **synthetic**. The private v5 mix is broader
+and includes reviewed non-synthetic source material after direct identifiers
+and sensitive fields were redacted from model-use fields. It is not a public
+corpus of real user records.
+
+## Bias, Risks, And Limitations
+
+This adapter is built on `openai/privacy-filter` and inherits the upstream
+model's bias, risk, and limitation profile. The notes below summarize and
+extend that profile for the Naija research preview. Consult the upstream
+model card for the authoritative description of base-model behavior.
+
+### Over-Reliance
+
+This adapter, like the base model, is a redaction and data-minimization aid.
+It is not an anonymization, compliance, or safety guarantee. Treating its
+output as a blanket anonymization claim risks missing the privacy objectives
+the system is being deployed to support. Use it as one layer in a
+privacy-by-design pipeline alongside policy controls, access controls,
+logging discipline, and human review where mistakes have material impact.
+The model detects spans; it does not by itself enforce retention, access
+control, consent, or data-subject rights.
+
+### Static Label Policy
+
+The model only detects spans that match its trained label taxonomy.
+Real-world privacy policies vary, and label boundaries appropriate for one
+organization may not be appropriate for another. Adjusting the policy
+requires further finetuning, not runtime configuration. The Naija adapter
+shifts boundaries for Nigerian-domain identifiers (NIN, BVN, NUBAN account
+numbers, voter card, driver license, passport, addresses, phone formats),
+but does not introduce a runtime policy configuration mechanism.
+
+### Domain And Language Coverage
+
+Performance can drop on:
+
+- non-English text and non-Latin scripts;
+- naming patterns or identifier formats not represented in training data;
+- domains outside the evaluated private mix, including unseen OCR layouts,
+  noisy chat logs, code-switched multilingual text, and organization-specific
+  record formats.
+
+The private evaluation mix cannot fully represent every deployment
+distribution. High typed-span F1 on the included splits should not be read as
+evidence of production readiness on all real records.
+
+### Failure Modes
+
+Like all models, this adapter can make mistakes. Common failure modes
+include:
+
+- under-detection of uncommon personal names, regional naming conventions,
+  initials, or honorific-heavy references;
+- over-redaction of organizations, locations, or common nouns when local
+  context is ambiguous;
+- fragmented or shifted span boundaries in mixed-format text, long
+  documents, or text with heavy punctuation and layout artifacts;
+- structured-identifier ambiguity - a numeric string may be a NIN, BVN,
+  account number, invoice number, order ID, or unrelated code, and the
+  model cannot always disambiguate without surrounding context;
+- missed secrets for novel credential formats, project-specific token
+  patterns, or secrets split across surrounding syntax;
+- over-redaction of benign high-entropy strings, hashes, placeholders, sample
+  credentials, synthetic examples, dates, checksums, and routing IDs that
+  resemble real secrets or identity numbers.
+
+Deterministic span postprocessing (`span_postprocess.py` in the source repo)
+reduces some boundary and known-format failures, but it is tuned for the
+synthetic Naija release. Applied outside that distribution it may itself
+introduce false positives.
+
+These failure modes can interact with demographic, regional, and domain
+variation. Names and identifiers underrepresented in the training data, or
+that follow conventions different from the dominant training distribution,
+are more likely to be missed or inconsistently bounded.
+
+### High-Risk Deployment Caution
+
+Additional caution is warranted in medical, legal, financial, human
+resources, education, and government workflows. In these settings, both
+false negatives and false positives can be costly: missed spans may expose
+sensitive information, while excess masking can remove material context
+needed for review, auditing, or downstream decisions. Do not use this adapter
+as the only control in such workflows.
+
+### Recommendations
+
+- Use the model as part of a privacy-by-design pipeline, not as a standalone
+  anonymization claim.
+- Evaluate on representative in-domain data under local policy before
+  production use.
+- Add deterministic filters for high-precision structured IDs where local
+  policy allows it.
+- Finetune further when your policy boundaries differ from the trained
+  taxonomy or when hard-negative precision matters.
+- Preserve human review paths for high-sensitivity workflows.
+
+## Privacy And Safety
+
+The public example bundle is synthetic and should not intentionally contain
+real personal data. Before publishing any new dataset, predictions, logs, or
+eval artifacts derived from this adapter, inspect them for accidental real PII
+or secrets.
+
+Do not publish:
+
+- raw real records;
+- production prompts, logs, tickets, emails, or support transcripts
+  containing personal data;
+- API keys, access tokens, cookies, or credentials;
+- model outputs that include unredacted real PII from private systems;
+- raw private/internal prediction JSONL or configs containing absolute paths,
+  private dataset IDs, or temporary directory names.
+
+## Citation And Attribution
+
+If you use this adapter, please cite both the base model and this repository.
+Preserve the adapter repo ID, dataset version, code commit, and evaluation
+artifact commit in experiment reports so results are reproducible.
+
+```bibtex
+@misc{egonu2026privacyfilternigeria,
+  author = {Egonu Narcisse},
+  title  = {Privacy Filter Nigeria LoRA (v0.1 research preview)},
+  year   = {2026},
+  url    = {https://github.com/iamNarcisse/naija-privacy-filter},
+  note   = {Adapter on top of openai/privacy-filter}
+}
+```
+
+## License
 
-The adapter detects spans from its trained label taxonomy and can miss or
-misclassify PII outside the training distribution. Do not use it as the only
-control for regulatory, legal, medical, financial, or irreversible privacy
-decisions. Evaluate on representative in-domain data before deployment.
+This adapter is released under the Apache License, Version 2.0. The base
+model `openai/privacy-filter` is governed by its own license; consult the
+upstream model card for terms.