enosislabs/matex-privacy-sentinel-v0.15-onnx

MaTE X Privacy Sentinel v0.15 ONNX — openai/privacy-filter fine-tune for code & AI secrets

Fine-tuned openai/privacy-filter for developer-focused PII and secret detection across 23 domain-specific categories built for code review, AI agent memory inspection, and repository scanning.

• Base model: openai/privacy-filter — 1.5B-parameter MoE (50M active per token), BIOES token-classification head
• Task: Token classification for PII and secret detection (BIOES scheme)
• Training data: 21,500 synthetic developer-domain examples (code snippets, API configs, database URIs, stack traces, agent prompts)
• Held-out eval: 1,075 examples / 133,459 tokens, label-stratified
• Recipe: opf train — full fine-tune, AdamW, lr=1e-4, 6 epochs, bf16
• Labels: 23 domain-specific categories → BIOES classes (O + 23 × B/I/E/S)
• Training hardware: NVIDIA L40S (48 GB VRAM) via Modal

The base model ships with 8 coarse PII categories. This model replaces that vocabulary with a developer and AI-native label space — repo_secret, api_key, database_uri, agent_memory_sensitive, stacktrace_sensitive, cloud_credential, and so on — matching what MaTE X code review and AI pipeline monitoring actually encounters.

This model is developed as part of the MaTE X platform by Enosis Labs. The name will be shortened in a future release.

---

Quick Start

With opf — OpenAI's official CLI

pip install 'opf @ git+https://github.com/openai/privacy-filter.git'

opf redact \
  --checkpoint enosis-labs/mate-x-privacy-sentinel-openai-privacy-filter-finetuned \
  --text "DATABASE_URL=postgresql://admin:s3cr3t@db.internal:5432/prod"

With transformers directly

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

model_id = "enosis-labs/mate-x-privacy-sentinel-openai-privacy-filter-finetuned"

tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForTokenClassification.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16
).to("cuda")
model.eval()

text = 'const db = new Client({ url: "postgresql://admin:s3cr3t@db.prod:5432/app" })'
enc = tok(text, return_tensors="pt").to("cuda")

with torch.no_grad():
    logits = model(**enc).logits.argmax(-1).cpu()[0].tolist()

id2label = {int(k): v for k, v in model.config.id2label.items()}
tokens = tok.convert_ids_to_tokens(enc["input_ids"][0].cpu().tolist())

for t, l in zip(tokens, logits):
    if l != 0:
        print(f"{t}\t{id2label[l]}")

Note: For best results use Viterbi decoding (not argmax) — opf does this by default. Raw argmax may show slightly more boundary errors on long secrets.

CPU inference (Transformers.js / Electron)

import { pipeline, env } from '@huggingface/transformers';

env.allowLocalModels = true;
env.localModelPath = './models/';

const detector = await pipeline(
  'token-classification',
  'mate-x-privacy-sentinel',   // local ONNX export
  { dtype: 'q8', device: 'cpu', aggregation_strategy: 'simple' }
);

const results = await detector(codeSnippet);
// results[i] = { entity_group, word, score, start, end }

---

Performance

Evaluated with opf eval --decode-mode viterbi --eval-mode typed on the held-out test set (1,075 examples, 133,459 tokens).

Global Metrics

Metric	Value
detection.f1	0.9871
detection.precision	0.9858
detection.recall	0.9885
span.f1	0.9541
token_accuracy	0.9740
loss	0.1230
inference speed	8,544 tokens/s

Improvement over Base Model (V3 → v0.15)

Metric	Base (V3)	v0.15	Delta
detection.f1	0.9747	0.9871	+1.24 pts
span.f1	0.9196	0.9541	+3.45 pts
token_accuracy	0.9520	0.9740	+2.20 pts
loss	0.2157	0.1230	-43.0%

Per-Class Span F1

Label	Precision	Recall	F1
🟢 private_url	1.0000	1.0000	1.0000
🟢 agent_memory_sensitive	1.0000	1.0000	1.0000
🟢 payment_token	0.9865	0.9932	0.9898
🟢 session_cookie	0.9895	0.9844	0.9869
🟢 database_uri	0.9899	0.9801	0.9850
🟢 prompt_sensitive	0.9831	0.9667	0.9748
🟢 cloud_credential	0.9707	0.9759	0.9733
🟢 repo_secret	1.0000	0.9423	0.9703
🟢 private_phone	0.9394	1.0000	0.9688
🟢 personal_document_id	0.9847	0.9699	0.9773
🟢 customer_data	0.9805	0.9264	0.9527
🟢 auth_token	0.9301	0.9492	0.9395
🟢 api_key	0.9553	0.9446	0.9499
🟢 internal_url	0.9141	0.9582	0.9356
🟢 private_file_path	0.9148	0.9198	0.9173
🟢 secret	0.9258	0.9231	0.9244
🟢 private_address	0.9615	0.9630	0.9623
🟢 workspace_identity	0.9140	0.9091	0.9115
🟢 stacktrace_sensitive	1.0000	0.8333	0.9091
🟢 private_date	0.8235	1.0000	0.9032
🟢 private_person	0.8537	0.9000	0.8762
🟢 private_email	0.8627	0.9362	0.8980
🟡 account_number	0.9412	0.7895	0.8587

Strong labels (F1 ≥ 0.90): 20 / 23 · Acceptable (F1 0.80–0.89): 3 / 23 · Weak (F1 < 0.80): 0 / 23

---

Label Space (23 Categories)

Category	Labels	Typical examples
Credentials & Secrets	api_key, auth_token, secret, repo_secret, cloud_credential, session_cookie, payment_token	API keys, Bearer tokens, .env values, GitHub PATs, AWS keys, Stripe tokens
Infrastructure	database_uri, internal_url, private_url, private_file_path	PostgreSQL URIs, Redis URLs, internal endpoints, absolute paths
Identity	private_person, private_email, private_phone, private_address, personal_document_id, account_number, private_date	Names, emails, phone numbers, ID cards, PAN numbers
AI / Agent	agent_memory_sensitive, prompt_sensitive, stacktrace_sensitive	Conversation memory, system prompts, full stack traces with locals
Platform	workspace_identity, customer_data	Org IDs, tenant slugs, user records embedded in code

---

Intended Use

This model is designed for:

• Code review pipelines — scan PR diffs and commits for accidentally committed secrets or PII before merge
• AI agent monitoring — detect sensitive data in agent memory, tool call outputs, and LLM context windows
• Repository auditing — bulk scan codebases for historical leaks
• Developer IDE integration — real-time inline warnings (see MaTE X / Aether)

---

Limitations

• Training data is synthetic. Real-world codebases may contain surface forms not represented in training data. For high-stakes deployments, collect a domain-specific eval set and re-calibrate thresholds.
• S-private_person (single-token person names) has F1 0.000 in the current release. Multi-token person names are detected correctly; single-token-only names (e.g., "Ada") will be missed. This is a known gap being addressed in the next training iteration with additional S-* examples.
• account_number F1 is 0.8587 — this class has few test examples (19 ground truth spans), so variance is high. Qualitative precision remains strong (0.9412).
• English-first. Training data is predominantly English. Performance on non-English code comments and string literals is not guaranteed.
• Not a substitute for secrets scanning tools (e.g., git-secrets, truffleHog). This model complements regex-based tools with contextual understanding; it does not replace them.
• Not legal compliance advice. Use alongside a governance layer and human review for regulatory contexts (GDPR, HIPAA, PCI-DSS, etc.).

---

Training Details

Parameter	Value
Base model	openai/privacy-filter
Training examples	21,500
Eval examples	1,075 (133,459 tokens)
Optimizer	AdamW
Learning rate	1e-4
Epochs	6 (best checkpoint at ep3, val_loss=0.1024)
Precision	bf16
Hardware	NVIDIA L40S 48 GB (Modal)
Inference speed	8,544 tokens/s (eval), 8,851 tokens/s (forward only)
BIOES classes	93 (1 O + 23 × B/I/E/S)

Head initialization: opf's default copy-from-matching-base strategy. Labels with exact matches in the base model (O, B/I/E/S-account_number, B/I/E/S-private_person, etc.) were copied directly; remaining classes were initialized from semantically adjacent base rows and fine-tuned end-to-end.

---

Credits & Acknowledgements

This model wouldn't exist without the following open-source releases — sincere thanks to both teams:

• OpenAI for open-sourcing the Privacy Filter — architecture, modeling code, and the opf training/eval CLI. Everything here is a fine-tune on top of that release.
• MaTE X / Enosis Labs for the developer-domain training dataset and domain label design.

---

License

Apache 2.0 — same as the base model openai/privacy-filter.

---

Citation

@misc{enosis_mate_x_privacy_sentinel_2026,
  author       = {Enosis Labs, Inc.},
  title        = {{MaTE X Privacy Sentinel}: openai/privacy-filter fine-tune for code \& AI secrets},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/enosislabs/matex-privacy-sentinel-v0.15-onnx}}
}

@misc{openai_privacy_filter_2025,
  author       = {OpenAI},
  title        = {{openai/privacy-filter}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/openai/privacy-filter}}
}