Initial publish of OpenMed/privacy-filter-multilingual. Renamed from privacy-filter-openmed-pii54-multilingual; clean README matching OpenMed/privacy-filter-nemotron format. Training artifact JSONs intentionally not redistributed here.

Browse files

Files changed (6) hide show

.gitattributes +1 -0
README.md +211 -0
config.json +770 -0
model.safetensors +3 -0
tokenizer.json +3 -0
tokenizer_config.json +11 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,211 @@

+---
+license: other
+library_name: transformers
+base_model: openai/privacy-filter
+datasets:
+  - ai4privacy/pii-masking-200k
+  - ai4privacy/pii-masking-400k
+  - ai4privacy/open-pii-masking-500k-ai4privacy
+pipeline_tag: token-classification
+tags:
+  - token-classification
+  - pii
+  - ner
+  - privacy
+  - redaction
+  - multilingual
+  - openmed
+  - openai-privacy-filter
+language:
+  - ar
+  - bn
+  - de
+  - en
+  - es
+  - fr
+  - hi
+  - it
+  - ja
+  - ko
+  - nl
+  - pt
+  - te
+  - tr
+  - vi
+  - zh
+---
+# privacy-filter-multilingual
+Fine-tuned [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter)
+for **fine-grained PII extraction** across **54 categories** in **16 languages**.
+- **Base model**: [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) — 1.4B-parameter MoE (50M active per token), BIOES token-classification head
+- **Task**: Token classification for PII detection (BIOES scheme)
+- **Languages (16)**: Arabic, Bengali, Chinese, Dutch, English, French, German, Hindi, Italian, Japanese, Korean, Portuguese, Spanish, Telugu, Turkish, Vietnamese
+- **Training data**: Multilingual mix from [AI4Privacy](https://huggingface.co/ai4privacy) — `pii-masking-200k`, `pii-masking-400k`, and `open-pii-masking-500k-ai4privacy`, language-balanced
+- **Recipe**: `opf train` (OpenAI's official fine-tuning CLI) — full fine-tune, AdamW, balanced language sampling, 5 epochs, bf16
+- **Labels**: 54 PII categories → 217 BIOES classes (1 `O` + 54 × B/I/E/S)
+The base model ships with 8 coarse PII categories and English-only training. This
+model trades that for a **6.75× more granular vocabulary** spanning identity,
+contact, address, financial, vehicle, digital, and crypto labels — all evaluated
+across 16 languages.
+> **Family at a glance.** Same architecture, three runtimes:
+> - **PyTorch (this repo)** — CPU + CUDA, anywhere transformers runs.
+> - **MLX BF16** — [`OpenMed/privacy-filter-multilingual-mlx`](https://huggingface.co/OpenMed/privacy-filter-multilingual-mlx) — Apple Silicon, full precision.
+> - **MLX 8-bit** — [`OpenMed/privacy-filter-multilingual-mlx-8bit`](https://huggingface.co/OpenMed/privacy-filter-multilingual-mlx-8bit) — Apple Silicon, smaller + faster.
+## Quick start
+### With [OpenMed](https://github.com/maziyarpanahi/openmed) — recommended
+OpenMed gives you `extract_pii()` / `deidentify()` with built-in BIOES Viterbi
+decoding, span refinement, and a Faker-backed obfuscation engine. Same call
+on every host — Apple Silicon picks up MLX automatically; everywhere else uses
+this PyTorch checkpoint.
+```bash
+pip install -U "openmed[hf]"
+```
+```python
+from openmed import extract_pii, deidentify
+text = (
+    "Patient Sarah Johnson (DOB 03/15/1985), MRN 4872910, "
+    "phone 415-555-0123, email sarah.johnson@example.com."
+)
+# Extract grouped entity spans
+result = extract_pii(text, model_name="OpenMed/privacy-filter-multilingual")
+for ent in result.entities:
+    print(f"{ent.label:30s} {ent.text!r}  conf={ent.confidence:.2f}")
+# De-identify with any of the supported methods
+masked   = deidentify(text, method="mask",   model_name="OpenMed/privacy-filter-multilingual")
+removed  = deidentify(text, method="remove", model_name="OpenMed/privacy-filter-multilingual")
+hashed   = deidentify(text, method="hash",   model_name="OpenMed/privacy-filter-multilingual")
+# Faker-backed locale-aware obfuscation, deterministic with consistent=True+seed
+fake = deidentify(
+    text,
+    method="replace",
+    model_name="OpenMed/privacy-filter-multilingual",
+    consistent=True,
+    seed=42,
+)
+print(fake.deidentified_text)
+```
+`OpenMed/privacy-filter-multilingual-mlx*` model names also work in the same
+`extract_pii()` / `deidentify()` calls — on a non-Apple-Silicon host they
+automatically fall back to **this PyTorch checkpoint** with a one-time warning.
+So you can ship MLX names in code and still run on Linux/Windows.
+The OpenMed wrapper passes `trust_remote_code=True` for you, runs the model's
+own BIOES Viterbi decoder, and skips OpenMed's regex smart-merging (the model
+already produces clean spans).
+## Label space (54 categories)
+| Category | Typical examples |
+|---|---|
+| **Identity** | `FIRSTNAME`, `MIDDLENAME`, `LASTNAME`, `PREFIX`, `AGE`, `GENDER`, `SEX`, `EYECOLOR`, `HEIGHT`, `USERNAME`, `OCCUPATION`, `JOBTITLE`, `JOBDEPARTMENT`, `ORGANIZATION`, `USERAGENT` |
+| **Contact** | `EMAIL`, `PHONE`, `URL` |
+| **Address** | `STREET`, `BUILDINGNUMBER`, `SECONDARYADDRESS`, `CITY`, `COUNTY`, `STATE`, `ZIPCODE`, `GPSCOORDINATES`, `ORDINALDIRECTION` |
+| **Dates & time** | `DATE`, `DATEOFBIRTH`, `TIME` |
+| **Government IDs** | `SSN` |
+| **Financial** | `ACCOUNTNAME`, `BANKACCOUNT`, `IBAN`, `BIC`, `CREDITCARD`, `CREDITCARDISSUER`, `CVV`, `PIN`, `MASKEDNUMBER`, `AMOUNT`, `CURRENCY`, `CURRENCYCODE`, `CURRENCYNAME`, `CURRENCYSYMBOL` |
+| **Crypto** | `BITCOINADDRESS`, `ETHEREUMADDRESS`, `LITECOINADDRESS` |
+| **Vehicle** | `VIN`, `VRM` |
+| **Digital** | `IPADDRESS`, `MACADDRESS`, `IMEI` |
+| **Auth** | `PASSWORD` |
+The output space is `O` plus `B-`, `I-`, `E-`, `S-` for each of the 54 categories
+(4 × 54 + 1 = 217). The `id2label` mapping is shipped with the model.
+## Limitations & intended use
+- **Multilingual but uneven.** Strongest on languages with rich PII training
+  data (German, Spanish, French, Italian, Hindi, Telugu, English). CJK languages
+  (Japanese, Korean, Chinese) and some morphologically-marked low-resource
+  languages remain the main bottleneck on the current training mix.
+- **Synthetic training data.** The AI4Privacy datasets are template-synthesized;
+  real clinical notes, legal documents, and web text may show different
+  surface forms. For high-stakes deployments, collect a domain-specific eval
+  set and re-calibrate thresholds.
+- **Not a substitute for legal compliance review.** Use alongside a governance
+  layer (human review, deterministic regex pre-filters, etc.).
+- **Not a clinical PHI model.** Healthcare-specific PHI and clinical entity
+  training is planned as a separate branch.
+**Head initialization**: `opf`'s default "copy-from-matching-base" head init.
+Of the 217 new BIOES classes, the few with exact base-vocabulary matches
+(`O`, `B/I/E/S-account_name`, etc.) were copied directly; the rest were copied
+from semantically-adjacent coarse rows and fine-tuned end-to-end.
+**Router**: base model has 128 MoE experts per layer with top-4 routing.
+Routers were kept trainable during full fine-tuning; no collapse was observed.
+## Credits & Acknowledgements
+This model wouldn't exist without two open-source releases — sincere thanks
+to both teams:
+- **OpenAI** for [open-sourcing the Privacy Filter](https://huggingface.co/openai/privacy-filter)
+  (architecture, modeling code, and `opf` training/eval CLI). Everything in
+  this repo is a fine-tune on top of that release.
+- **AI4Privacy** for releasing the multilingual PII masking datasets used as
+  training data:
+  [`pii-masking-200k`](https://huggingface.co/datasets/ai4privacy/pii-masking-200k),
+  [`pii-masking-400k`](https://huggingface.co/datasets/ai4privacy/pii-masking-400k),
+  [`open-pii-masking-500k-ai4privacy`](https://huggingface.co/datasets/ai4privacy/open-pii-masking-500k-ai4privacy).
+Additional thanks to the **HuggingFace** team for the `transformers` /
+`huggingface_hub` ecosystem this model ships through.
+## License
+Inherits the licensing of its inputs. The base model `openai/privacy-filter`
+is Apache 2.0; the AI4Privacy datasets carry their own terms — please consult
+those sources before redistribution.
+## Citation
+If you use this model, please cite **this model**, the organization behind it
+(**OpenMed**), and the upstream base model + datasets:
+```bibtex
+@misc{openmed_privacy_filter_multilingual_2026,
+  author       = {OpenMed},
+  title        = {{OpenMed/privacy-filter-multilingual}: multilingual fine-grained PII extraction across 16 languages and 54 categories},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/OpenMed/privacy-filter-multilingual}}
+}
+@misc{openmed_2026,
+  author       = {OpenMed},
+  title        = {{OpenMed}: open models and resources for healthcare NLP},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/OpenMed}}
+}
+@misc{openai_privacy_filter_2025,
+  author       = {OpenAI},
+  title        = {{openai/privacy-filter}},
+  year         = {2025},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/openai/privacy-filter}}
+}
+@misc{ai4privacy_pii_masking,
+  author       = {AI4Privacy},
+  title        = {{AI4Privacy PII Masking Datasets}},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/ai4privacy}}
+}
+```

config.json ADDED Viewed

	@@ -0,0 +1,770 @@

+{
+  "architectures": [
+    "OpenAIPrivacyFilterForTokenClassification"
+  ],
+  "attention_bias": true,
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "classifier_dropout": 0.0,
+  "default_n_ctx": 128000,
+  "dtype": "bfloat16",
+  "eos_token_id": 199999,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 640,
+  "id2label": {
+    "0": "O",
+    "1": "B-ACCOUNTNAME",
+    "2": "I-ACCOUNTNAME",
+    "3": "E-ACCOUNTNAME",
+    "4": "S-ACCOUNTNAME",
+    "5": "B-AGE",
+    "6": "I-AGE",
+    "7": "E-AGE",
+    "8": "S-AGE",
+    "9": "B-AMOUNT",
+    "10": "I-AMOUNT",
+    "11": "E-AMOUNT",
+    "12": "S-AMOUNT",
+    "13": "B-BANKACCOUNT",
+    "14": "I-BANKACCOUNT",
+    "15": "E-BANKACCOUNT",
+    "16": "S-BANKACCOUNT",
+    "17": "B-BIC",
+    "18": "I-BIC",
+    "19": "E-BIC",
+    "20": "S-BIC",
+    "21": "B-BITCOINADDRESS",
+    "22": "I-BITCOINADDRESS",
+    "23": "E-BITCOINADDRESS",
+    "24": "S-BITCOINADDRESS",
+    "25": "B-BUILDINGNUMBER",
+    "26": "I-BUILDINGNUMBER",
+    "27": "E-BUILDINGNUMBER",
+    "28": "S-BUILDINGNUMBER",
+    "29": "B-CITY",
+    "30": "I-CITY",
+    "31": "E-CITY",
+    "32": "S-CITY",
+    "33": "B-COUNTY",
+    "34": "I-COUNTY",
+    "35": "E-COUNTY",
+    "36": "S-COUNTY",
+    "37": "B-CREDITCARD",
+    "38": "I-CREDITCARD",
+    "39": "E-CREDITCARD",
+    "40": "S-CREDITCARD",
+    "41": "B-CREDITCARDISSUER",
+    "42": "I-CREDITCARDISSUER",
+    "43": "E-CREDITCARDISSUER",
+    "44": "S-CREDITCARDISSUER",
+    "45": "B-CURRENCY",
+    "46": "I-CURRENCY",
+    "47": "E-CURRENCY",
+    "48": "S-CURRENCY",
+    "49": "B-CURRENCYCODE",
+    "50": "I-CURRENCYCODE",
+    "51": "E-CURRENCYCODE",
+    "52": "S-CURRENCYCODE",
+    "53": "B-CURRENCYNAME",
+    "54": "I-CURRENCYNAME",
+    "55": "E-CURRENCYNAME",
+    "56": "S-CURRENCYNAME",
+    "57": "B-CURRENCYSYMBOL",
+    "58": "I-CURRENCYSYMBOL",
+    "59": "E-CURRENCYSYMBOL",
+    "60": "S-CURRENCYSYMBOL",
+    "61": "B-CVV",
+    "62": "I-CVV",
+    "63": "E-CVV",
+    "64": "S-CVV",
+    "65": "B-DATE",
+    "66": "I-DATE",
+    "67": "E-DATE",
+    "68": "S-DATE",
+    "69": "B-DATEOFBIRTH",
+    "70": "I-DATEOFBIRTH",
+    "71": "E-DATEOFBIRTH",
+    "72": "S-DATEOFBIRTH",
+    "73": "B-EMAIL",
+    "74": "I-EMAIL",
+    "75": "E-EMAIL",
+    "76": "S-EMAIL",
+    "77": "B-ETHEREUMADDRESS",
+    "78": "I-ETHEREUMADDRESS",
+    "79": "E-ETHEREUMADDRESS",
+    "80": "S-ETHEREUMADDRESS",
+    "81": "B-EYECOLOR",
+    "82": "I-EYECOLOR",
+    "83": "E-EYECOLOR",
+    "84": "S-EYECOLOR",
+    "85": "B-FIRSTNAME",
+    "86": "I-FIRSTNAME",
+    "87": "E-FIRSTNAME",
+    "88": "S-FIRSTNAME",
+    "89": "B-GENDER",
+    "90": "I-GENDER",
+    "91": "E-GENDER",
+    "92": "S-GENDER",
+    "93": "B-GPSCOORDINATES",
+    "94": "I-GPSCOORDINATES",
+    "95": "E-GPSCOORDINATES",
+    "96": "S-GPSCOORDINATES",
+    "97": "B-HEIGHT",
+    "98": "I-HEIGHT",
+    "99": "E-HEIGHT",
+    "100": "S-HEIGHT",
+    "101": "B-IBAN",
+    "102": "I-IBAN",
+    "103": "E-IBAN",
+    "104": "S-IBAN",
+    "105": "B-IMEI",
+    "106": "I-IMEI",
+    "107": "E-IMEI",
+    "108": "S-IMEI",
+    "109": "B-IPADDRESS",
+    "110": "I-IPADDRESS",
+    "111": "E-IPADDRESS",
+    "112": "S-IPADDRESS",
+    "113": "B-JOBDEPARTMENT",
+    "114": "I-JOBDEPARTMENT",
+    "115": "E-JOBDEPARTMENT",
+    "116": "S-JOBDEPARTMENT",
+    "117": "B-JOBTITLE",
+    "118": "I-JOBTITLE",
+    "119": "E-JOBTITLE",
+    "120": "S-JOBTITLE",
+    "121": "B-LASTNAME",
+    "122": "I-LASTNAME",
+    "123": "E-LASTNAME",
+    "124": "S-LASTNAME",
+    "125": "B-LITECOINADDRESS",
+    "126": "I-LITECOINADDRESS",
+    "127": "E-LITECOINADDRESS",
+    "128": "S-LITECOINADDRESS",
+    "129": "B-MACADDRESS",
+    "130": "I-MACADDRESS",
+    "131": "E-MACADDRESS",
+    "132": "S-MACADDRESS",
+    "133": "B-MASKEDNUMBER",
+    "134": "I-MASKEDNUMBER",
+    "135": "E-MASKEDNUMBER",
+    "136": "S-MASKEDNUMBER",
+    "137": "B-MIDDLENAME",
+    "138": "I-MIDDLENAME",
+    "139": "E-MIDDLENAME",
+    "140": "S-MIDDLENAME",
+    "141": "B-OCCUPATION",
+    "142": "I-OCCUPATION",
+    "143": "E-OCCUPATION",
+    "144": "S-OCCUPATION",
+    "145": "B-ORDINALDIRECTION",
+    "146": "I-ORDINALDIRECTION",
+    "147": "E-ORDINALDIRECTION",
+    "148": "S-ORDINALDIRECTION",
+    "149": "B-ORGANIZATION",
+    "150": "I-ORGANIZATION",
+    "151": "E-ORGANIZATION",
+    "152": "S-ORGANIZATION",
+    "153": "B-PASSWORD",
+    "154": "I-PASSWORD",
+    "155": "E-PASSWORD",
+    "156": "S-PASSWORD",
+    "157": "B-PHONE",
+    "158": "I-PHONE",
+    "159": "E-PHONE",
+    "160": "S-PHONE",
+    "161": "B-PIN",
+    "162": "I-PIN",
+    "163": "E-PIN",
+    "164": "S-PIN",
+    "165": "B-PREFIX",
+    "166": "I-PREFIX",
+    "167": "E-PREFIX",
+    "168": "S-PREFIX",
+    "169": "B-SECONDARYADDRESS",
+    "170": "I-SECONDARYADDRESS",
+    "171": "E-SECONDARYADDRESS",
+    "172": "S-SECONDARYADDRESS",
+    "173": "B-SEX",
+    "174": "I-SEX",
+    "175": "E-SEX",
+    "176": "S-SEX",
+    "177": "B-SSN",
+    "178": "I-SSN",
+    "179": "E-SSN",
+    "180": "S-SSN",
+    "181": "B-STATE",
+    "182": "I-STATE",
+    "183": "E-STATE",
+    "184": "S-STATE",
+    "185": "B-STREET",
+    "186": "I-STREET",
+    "187": "E-STREET",
+    "188": "S-STREET",
+    "189": "B-TIME",
+    "190": "I-TIME",
+    "191": "E-TIME",
+    "192": "S-TIME",
+    "193": "B-URL",
+    "194": "I-URL",
+    "195": "E-URL",
+    "196": "S-URL",
+    "197": "B-USERAGENT",
+    "198": "I-USERAGENT",
+    "199": "E-USERAGENT",
+    "200": "S-USERAGENT",
+    "201": "B-USERNAME",
+    "202": "I-USERNAME",
+    "203": "E-USERNAME",
+    "204": "S-USERNAME",
+    "205": "B-VIN",
+    "206": "I-VIN",
+    "207": "E-VIN",
+    "208": "S-VIN",
+    "209": "B-VRM",
+    "210": "I-VRM",
+    "211": "E-VRM",
+    "212": "S-VRM",
+    "213": "B-ZIPCODE",
+    "214": "I-ZIPCODE",
+    "215": "E-ZIPCODE",
+    "216": "S-ZIPCODE"
+  },
+  "initial_context_length": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 640,
+  "label2id": {
+    "O": 0,
+    "B-ACCOUNTNAME": 1,
+    "I-ACCOUNTNAME": 2,
+    "E-ACCOUNTNAME": 3,
+    "S-ACCOUNTNAME": 4,
+    "B-AGE": 5,
+    "I-AGE": 6,
+    "E-AGE": 7,
+    "S-AGE": 8,
+    "B-AMOUNT": 9,
+    "I-AMOUNT": 10,
+    "E-AMOUNT": 11,
+    "S-AMOUNT": 12,
+    "B-BANKACCOUNT": 13,
+    "I-BANKACCOUNT": 14,
+    "E-BANKACCOUNT": 15,
+    "S-BANKACCOUNT": 16,
+    "B-BIC": 17,
+    "I-BIC": 18,
+    "E-BIC": 19,
+    "S-BIC": 20,
+    "B-BITCOINADDRESS": 21,
+    "I-BITCOINADDRESS": 22,
+    "E-BITCOINADDRESS": 23,
+    "S-BITCOINADDRESS": 24,
+    "B-BUILDINGNUMBER": 25,
+    "I-BUILDINGNUMBER": 26,
+    "E-BUILDINGNUMBER": 27,
+    "S-BUILDINGNUMBER": 28,
+    "B-CITY": 29,
+    "I-CITY": 30,
+    "E-CITY": 31,
+    "S-CITY": 32,
+    "B-COUNTY": 33,
+    "I-COUNTY": 34,
+    "E-COUNTY": 35,
+    "S-COUNTY": 36,
+    "B-CREDITCARD": 37,
+    "I-CREDITCARD": 38,
+    "E-CREDITCARD": 39,
+    "S-CREDITCARD": 40,
+    "B-CREDITCARDISSUER": 41,
+    "I-CREDITCARDISSUER": 42,
+    "E-CREDITCARDISSUER": 43,
+    "S-CREDITCARDISSUER": 44,
+    "B-CURRENCY": 45,
+    "I-CURRENCY": 46,
+    "E-CURRENCY": 47,
+    "S-CURRENCY": 48,
+    "B-CURRENCYCODE": 49,
+    "I-CURRENCYCODE": 50,
+    "E-CURRENCYCODE": 51,
+    "S-CURRENCYCODE": 52,
+    "B-CURRENCYNAME": 53,
+    "I-CURRENCYNAME": 54,
+    "E-CURRENCYNAME": 55,
+    "S-CURRENCYNAME": 56,
+    "B-CURRENCYSYMBOL": 57,
+    "I-CURRENCYSYMBOL": 58,
+    "E-CURRENCYSYMBOL": 59,
+    "S-CURRENCYSYMBOL": 60,
+    "B-CVV": 61,
+    "I-CVV": 62,
+    "E-CVV": 63,
+    "S-CVV": 64,
+    "B-DATE": 65,
+    "I-DATE": 66,
+    "E-DATE": 67,
+    "S-DATE": 68,
+    "B-DATEOFBIRTH": 69,
+    "I-DATEOFBIRTH": 70,
+    "E-DATEOFBIRTH": 71,
+    "S-DATEOFBIRTH": 72,
+    "B-EMAIL": 73,
+    "I-EMAIL": 74,
+    "E-EMAIL": 75,
+    "S-EMAIL": 76,
+    "B-ETHEREUMADDRESS": 77,
+    "I-ETHEREUMADDRESS": 78,
+    "E-ETHEREUMADDRESS": 79,
+    "S-ETHEREUMADDRESS": 80,
+    "B-EYECOLOR": 81,
+    "I-EYECOLOR": 82,
+    "E-EYECOLOR": 83,
+    "S-EYECOLOR": 84,
+    "B-FIRSTNAME": 85,
+    "I-FIRSTNAME": 86,
+    "E-FIRSTNAME": 87,
+    "S-FIRSTNAME": 88,
+    "B-GENDER": 89,
+    "I-GENDER": 90,
+    "E-GENDER": 91,
+    "S-GENDER": 92,
+    "B-GPSCOORDINATES": 93,
+    "I-GPSCOORDINATES": 94,
+    "E-GPSCOORDINATES": 95,
+    "S-GPSCOORDINATES": 96,
+    "B-HEIGHT": 97,
+    "I-HEIGHT": 98,
+    "E-HEIGHT": 99,
+    "S-HEIGHT": 100,
+    "B-IBAN": 101,
+    "I-IBAN": 102,
+    "E-IBAN": 103,
+    "S-IBAN": 104,
+    "B-IMEI": 105,
+    "I-IMEI": 106,
+    "E-IMEI": 107,
+    "S-IMEI": 108,
+    "B-IPADDRESS": 109,
+    "I-IPADDRESS": 110,
+    "E-IPADDRESS": 111,
+    "S-IPADDRESS": 112,
+    "B-JOBDEPARTMENT": 113,
+    "I-JOBDEPARTMENT": 114,
+    "E-JOBDEPARTMENT": 115,
+    "S-JOBDEPARTMENT": 116,
+    "B-JOBTITLE": 117,
+    "I-JOBTITLE": 118,
+    "E-JOBTITLE": 119,
+    "S-JOBTITLE": 120,
+    "B-LASTNAME": 121,
+    "I-LASTNAME": 122,
+    "E-LASTNAME": 123,
+    "S-LASTNAME": 124,
+    "B-LITECOINADDRESS": 125,
+    "I-LITECOINADDRESS": 126,
+    "E-LITECOINADDRESS": 127,
+    "S-LITECOINADDRESS": 128,
+    "B-MACADDRESS": 129,
+    "I-MACADDRESS": 130,
+    "E-MACADDRESS": 131,
+    "S-MACADDRESS": 132,
+    "B-MASKEDNUMBER": 133,
+    "I-MASKEDNUMBER": 134,
+    "E-MASKEDNUMBER": 135,
+    "S-MASKEDNUMBER": 136,
+    "B-MIDDLENAME": 137,
+    "I-MIDDLENAME": 138,
+    "E-MIDDLENAME": 139,
+    "S-MIDDLENAME": 140,
+    "B-OCCUPATION": 141,
+    "I-OCCUPATION": 142,
+    "E-OCCUPATION": 143,
+    "S-OCCUPATION": 144,
+    "B-ORDINALDIRECTION": 145,
+    "I-ORDINALDIRECTION": 146,
+    "E-ORDINALDIRECTION": 147,
+    "S-ORDINALDIRECTION": 148,
+    "B-ORGANIZATION": 149,
+    "I-ORGANIZATION": 150,
+    "E-ORGANIZATION": 151,
+    "S-ORGANIZATION": 152,
+    "B-PASSWORD": 153,
+    "I-PASSWORD": 154,
+    "E-PASSWORD": 155,
+    "S-PASSWORD": 156,
+    "B-PHONE": 157,
+    "I-PHONE": 158,
+    "E-PHONE": 159,
+    "S-PHONE": 160,
+    "B-PIN": 161,
+    "I-PIN": 162,
+    "E-PIN": 163,
+    "S-PIN": 164,
+    "B-PREFIX": 165,
+    "I-PREFIX": 166,
+    "E-PREFIX": 167,
+    "S-PREFIX": 168,
+    "B-SECONDARYADDRESS": 169,
+    "I-SECONDARYADDRESS": 170,
+    "E-SECONDARYADDRESS": 171,
+    "S-SECONDARYADDRESS": 172,
+    "B-SEX": 173,
+    "I-SEX": 174,
+    "E-SEX": 175,
+    "S-SEX": 176,
+    "B-SSN": 177,
+    "I-SSN": 178,
+    "E-SSN": 179,
+    "S-SSN": 180,
+    "B-STATE": 181,
+    "I-STATE": 182,
+    "E-STATE": 183,
+    "S-STATE": 184,
+    "B-STREET": 185,
+    "I-STREET": 186,
+    "E-STREET": 187,
+    "S-STREET": 188,
+    "B-TIME": 189,
+    "I-TIME": 190,
+    "E-TIME": 191,
+    "S-TIME": 192,
+    "B-URL": 193,
+    "I-URL": 194,
+    "E-URL": 195,
+    "S-URL": 196,
+    "B-USERAGENT": 197,
+    "I-USERAGENT": 198,
+    "E-USERAGENT": 199,
+    "S-USERAGENT": 200,
+    "B-USERNAME": 201,
+    "I-USERNAME": 202,
+    "E-USERNAME": 203,
+    "S-USERNAME": 204,
+    "B-VIN": 205,
+    "I-VIN": 206,
+    "E-VIN": 207,
+    "S-VIN": 208,
+    "B-VRM": 209,
+    "I-VRM": 210,
+    "E-VRM": 211,
+    "S-VRM": 212,
+    "B-ZIPCODE": 213,
+    "I-ZIPCODE": 214,
+    "E-ZIPCODE": 215,
+    "S-ZIPCODE": 216
+  },
+  "max_position_embeddings": 131072,
+  "model_type": "openai_privacy_filter",
+  "num_attention_heads": 14,
+  "num_experts_per_tok": 4,
+  "num_hidden_layers": 8,
+  "num_key_value_heads": 2,
+  "num_local_experts": 128,
+  "output_router_logits": false,
+  "pad_token_id": 199999,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "beta_fast": 32.0,
+    "beta_slow": 1.0,
+    "factor": 32.0,
+    "original_max_position_embeddings": 4096,
+    "rope_theta": 150000.0,
+    "rope_type": "yarn",
+    "truncate": false
+  },
+  "router_aux_loss_coef": 0.001,
+  "sliding_window": 128,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.6.0.dev0",
+  "use_cache": true,
+  "vocab_size": 200064,
+  "transformers.js_config": {
+    "use_external_data_format": {
+      "model.onnx": 3,
+      "model_fp16.onnx": 2,
+      "model": 1
+    }
+  },
+  "num_labels": 217,
+  "opf_metadata": {
+    "category_version": "openmed_pii54_multi_v1",
+    "encoding": "o200k_base",
+    "span_class_names": [
+      "O",
+      "ACCOUNTNAME",
+      "AGE",
+      "AMOUNT",
+      "BANKACCOUNT",
+      "BIC",
+      "BITCOINADDRESS",
+      "BUILDINGNUMBER",
+      "CITY",
+      "COUNTY",
+      "CREDITCARD",
+      "CREDITCARDISSUER",
+      "CURRENCY",
+      "CURRENCYCODE",
+      "CURRENCYNAME",
+      "CURRENCYSYMBOL",
+      "CVV",
+      "DATE",
+      "DATEOFBIRTH",
+      "EMAIL",
+      "ETHEREUMADDRESS",
+      "EYECOLOR",
+      "FIRSTNAME",
+      "GENDER",
+      "GPSCOORDINATES",
+      "HEIGHT",
+      "IBAN",
+      "IMEI",
+      "IPADDRESS",
+      "JOBDEPARTMENT",
+      "JOBTITLE",
+      "LASTNAME",
+      "LITECOINADDRESS",
+      "MACADDRESS",
+      "MASKEDNUMBER",
+      "MIDDLENAME",
+      "OCCUPATION",
+      "ORDINALDIRECTION",
+      "ORGANIZATION",
+      "PASSWORD",
+      "PHONE",
+      "PIN",
+      "PREFIX",
+      "SECONDARYADDRESS",
+      "SEX",
+      "SSN",
+      "STATE",
+      "STREET",
+      "TIME",
+      "URL",
+      "USERAGENT",
+      "USERNAME",
+      "VIN",
+      "VRM",
+      "ZIPCODE"
+    ],
+    "ner_class_names": [
+      "O",
+      "B-ACCOUNTNAME",
+      "I-ACCOUNTNAME",
+      "E-ACCOUNTNAME",
+      "S-ACCOUNTNAME",
+      "B-AGE",
+      "I-AGE",
+      "E-AGE",
+      "S-AGE",
+      "B-AMOUNT",
+      "I-AMOUNT",
+      "E-AMOUNT",
+      "S-AMOUNT",
+      "B-BANKACCOUNT",
+      "I-BANKACCOUNT",
+      "E-BANKACCOUNT",
+      "S-BANKACCOUNT",
+      "B-BIC",
+      "I-BIC",
+      "E-BIC",
+      "S-BIC",
+      "B-BITCOINADDRESS",
+      "I-BITCOINADDRESS",
+      "E-BITCOINADDRESS",
+      "S-BITCOINADDRESS",
+      "B-BUILDINGNUMBER",
+      "I-BUILDINGNUMBER",
+      "E-BUILDINGNUMBER",
+      "S-BUILDINGNUMBER",
+      "B-CITY",
+      "I-CITY",
+      "E-CITY",
+      "S-CITY",
+      "B-COUNTY",
+      "I-COUNTY",
+      "E-COUNTY",
+      "S-COUNTY",
+      "B-CREDITCARD",
+      "I-CREDITCARD",
+      "E-CREDITCARD",
+      "S-CREDITCARD",
+      "B-CREDITCARDISSUER",
+      "I-CREDITCARDISSUER",
+      "E-CREDITCARDISSUER",
+      "S-CREDITCARDISSUER",
+      "B-CURRENCY",
+      "I-CURRENCY",
+      "E-CURRENCY",
+      "S-CURRENCY",
+      "B-CURRENCYCODE",
+      "I-CURRENCYCODE",
+      "E-CURRENCYCODE",
+      "S-CURRENCYCODE",
+      "B-CURRENCYNAME",
+      "I-CURRENCYNAME",
+      "E-CURRENCYNAME",
+      "S-CURRENCYNAME",
+      "B-CURRENCYSYMBOL",
+      "I-CURRENCYSYMBOL",
+      "E-CURRENCYSYMBOL",
+      "S-CURRENCYSYMBOL",
+      "B-CVV",
+      "I-CVV",
+      "E-CVV",
+      "S-CVV",
+      "B-DATE",
+      "I-DATE",
+      "E-DATE",
+      "S-DATE",
+      "B-DATEOFBIRTH",
+      "I-DATEOFBIRTH",
+      "E-DATEOFBIRTH",
+      "S-DATEOFBIRTH",
+      "B-EMAIL",
+      "I-EMAIL",
+      "E-EMAIL",
+      "S-EMAIL",
+      "B-ETHEREUMADDRESS",
+      "I-ETHEREUMADDRESS",
+      "E-ETHEREUMADDRESS",
+      "S-ETHEREUMADDRESS",
+      "B-EYECOLOR",
+      "I-EYECOLOR",
+      "E-EYECOLOR",
+      "S-EYECOLOR",
+      "B-FIRSTNAME",
+      "I-FIRSTNAME",
+      "E-FIRSTNAME",
+      "S-FIRSTNAME",
+      "B-GENDER",
+      "I-GENDER",
+      "E-GENDER",
+      "S-GENDER",
+      "B-GPSCOORDINATES",
+      "I-GPSCOORDINATES",
+      "E-GPSCOORDINATES",
+      "S-GPSCOORDINATES",
+      "B-HEIGHT",
+      "I-HEIGHT",
+      "E-HEIGHT",
+      "S-HEIGHT",
+      "B-IBAN",
+      "I-IBAN",
+      "E-IBAN",
+      "S-IBAN",
+      "B-IMEI",
+      "I-IMEI",
+      "E-IMEI",
+      "S-IMEI",
+      "B-IPADDRESS",
+      "I-IPADDRESS",
+      "E-IPADDRESS",
+      "S-IPADDRESS",
+      "B-JOBDEPARTMENT",
+      "I-JOBDEPARTMENT",
+      "E-JOBDEPARTMENT",
+      "S-JOBDEPARTMENT",
+      "B-JOBTITLE",
+      "I-JOBTITLE",
+      "E-JOBTITLE",
+      "S-JOBTITLE",
+      "B-LASTNAME",
+      "I-LASTNAME",
+      "E-LASTNAME",
+      "S-LASTNAME",
+      "B-LITECOINADDRESS",
+      "I-LITECOINADDRESS",
+      "E-LITECOINADDRESS",
+      "S-LITECOINADDRESS",
+      "B-MACADDRESS",
+      "I-MACADDRESS",
+      "E-MACADDRESS",
+      "S-MACADDRESS",
+      "B-MASKEDNUMBER",
+      "I-MASKEDNUMBER",
+      "E-MASKEDNUMBER",
+      "S-MASKEDNUMBER",
+      "B-MIDDLENAME",
+      "I-MIDDLENAME",
+      "E-MIDDLENAME",
+      "S-MIDDLENAME",
+      "B-OCCUPATION",
+      "I-OCCUPATION",
+      "E-OCCUPATION",
+      "S-OCCUPATION",
+      "B-ORDINALDIRECTION",
+      "I-ORDINALDIRECTION",
+      "E-ORDINALDIRECTION",
+      "S-ORDINALDIRECTION",
+      "B-ORGANIZATION",
+      "I-ORGANIZATION",
+      "E-ORGANIZATION",
+      "S-ORGANIZATION",
+      "B-PASSWORD",
+      "I-PASSWORD",
+      "E-PASSWORD",
+      "S-PASSWORD",
+      "B-PHONE",
+      "I-PHONE",
+      "E-PHONE",
+      "S-PHONE",
+      "B-PIN",
+      "I-PIN",
+      "E-PIN",
+      "S-PIN",
+      "B-PREFIX",
+      "I-PREFIX",
+      "E-PREFIX",
+      "S-PREFIX",
+      "B-SECONDARYADDRESS",
+      "I-SECONDARYADDRESS",
+      "E-SECONDARYADDRESS",
+      "S-SECONDARYADDRESS",
+      "B-SEX",
+      "I-SEX",
+      "E-SEX",
+      "S-SEX",
+      "B-SSN",
+      "I-SSN",
+      "E-SSN",
+      "S-SSN",
+      "B-STATE",
+      "I-STATE",
+      "E-STATE",
+      "S-STATE",
+      "B-STREET",
+      "I-STREET",
+      "E-STREET",
+      "S-STREET",
+      "B-TIME",
+      "I-TIME",
+      "E-TIME",
+      "S-TIME",
+      "B-URL",
+      "I-URL",
+      "E-URL",
+      "S-URL",
+      "B-USERAGENT",
+      "I-USERAGENT",
+      "E-USERAGENT",
+      "S-USERAGENT",
+      "B-USERNAME",
+      "I-USERNAME",
+      "E-USERNAME",
+      "S-USERNAME",
+      "B-VIN",
+      "I-VIN",
+      "E-VIN",
+      "S-VIN",
+      "B-VRM",
+      "I-VRM",
+      "E-VRM",
+      "S-VRM",
+      "B-ZIPCODE",
+      "I-ZIPCODE",
+      "E-ZIPCODE",
+      "S-ZIPCODE"
+    ],
+    "inference_contract_version": 1
+  }
+}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f0b62052eaabe7918b0b1ba24564432902b4c4a3c61404cf2b494679d21e97b6
+size 2799225250

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0614fe83cadab421296e664e1f48f4261fa8fef6e03e63bb75c20f38e37d07d3
+size 27868174

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,11 @@

+{
+  "backend": "tokenizers",
+  "eos_token": "<|endoftext|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "TokenizersBackend"
+}