Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-2898/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-3220/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,296 @@
+---
+license: apache-2.0
+language:
+  - ko
+  - en
+tags:
+  - privacy-filter
+  - pii-detection
+  - token-classification
+  - korean
+  - lora
+  - openai-privacy-filter
+  - bioes
+base_model: openai/privacy-filter
+pipeline_tag: token-classification
+---
+
+# Privacy Filter — Korean
+
+Korean fine-tune of [OpenAI Privacy Filter](https://huggingface.co/openai/privacy-filter)
+for span-level PII detection. Adapted via **LoRA** on attention projections only —
+the base's sparse-MoE backbone (1.5B / 50M active params) is kept frozen, with
+just **~614k trainable parameters** (~0.04% of the model).
+
+**[Open Test Notebook](https://huggingface.co/FrameByFrame/privacy-filter-korean/blob/main/test_privacy_filter_ko.ipynb)** — load the model and run all examples interactively.
+
+## Capabilities
+
+| Category | Description | Example |
+|---|---|---|
+| `private_person` | Personal name (Korean / Western / handles) | 김민수, John Smith |
+| `private_address` | Physical / postal address | 서울특별시 강남구 테헤란로 123 |
+| `private_phone` | Phone number | 010-1234-5678 |
+| `private_email` | Email address | minsu@example.com |
+| `private_date` | Birthday / personally-identifying date | 1985년 3월 12일 |
+| `private_url` | Personal URL | github.com/minsu |
+| `account_number` | Bank, card, RRN, passport, etc. | 110-234-567890 |
+| `personal_handle` | Username / handle | @minsu_dev |
+| `ip_address` | IP address | 192.168.1.5 |
+
+## Benchmark Results
+
+Held-out KDPII Korean PII test set, span-level F1:
+
+| label | base | fine-tuned | Δ |
+|---|---|---|---|
+| `private_phone` | 0.65 | **1.00** | +0.35 |
+| `private_url` | 0.21 | **1.00** | +0.79 |
+| `private_email` | 0.86 | **1.00** | +0.14 |
+| `account_number` | 0.31 | **0.98** | +0.67 |
+| `private_date` | 0.00 | **0.90** | +0.90 |
+| `private_address` | 0.00 | **0.78** | +0.78 |
+| `private_person` | 0.06 | **0.69** | +0.63 |
+| **Overall** | — | — | **+0.58** |
+
+## Quick Start
+
+### Install
+
+```bash
+pip install transformers peft torch
+```
+
+### Load Model
+
+```python
+from transformers import AutoTokenizer, AutoModelForTokenClassification
+import torch
+
+MODEL_ID = "FrameByFrame/privacy-filter-korean"
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
+model = AutoModelForTokenClassification.from_pretrained(
+    MODEL_ID, trust_remote_code=True, torch_dtype=torch.bfloat16
+)
+model.eval()
+if torch.cuda.is_available():
+    model.cuda()
+```
+
+`trust_remote_code=True` is required because Privacy Filter ships a custom
+`OpenAIPrivacyFilterForTokenClassification` class (gpt-oss-style sparse MoE).
+
+### Inference
+
+The model emits per-token BIOES labels. The helper below decodes them into
+character-offset spans with simple constrained logic:
+
+```python
+def extract_pii(text: str, max_length: int = 512):
+    enc = tokenizer(
+        text,
+        truncation=True,
+        max_length=max_length,
+        return_offsets_mapping=True,
+        return_tensors="pt",
+    )
+    offsets = enc.pop("offset_mapping")[0].tolist()
+    enc = {k: v.to(model.device) for k, v in enc.items()}
+    with torch.no_grad():
+        logits = model(**enc).logits
+    pred_ids = logits.argmax(-1)[0].tolist()
+    id2label = model.config.id2label
+
+    spans = []
+    active = None  # (label, start, end)
+    for tok_idx, lid in enumerate(pred_ids):
+        label = id2label[int(lid)]
+        if label == "O":
+            if active is not None:
+                spans.append(active); active = None
+            continue
+        prefix, cat = label.split("-", 1)
+        c_start, c_end = offsets[tok_idx]
+        if prefix == "S":
+            if active is not None: spans.append(active); active = None
+            spans.append((cat, c_start, c_end))
+        elif prefix == "B":
+            if active is not None: spans.append(active)
+            active = (cat, c_start, c_end)
+        elif prefix in ("I", "E"):
+            if active and active[0] == cat:
+                active = (active[0], active[1], c_end)
+            else:
+                if active is not None: spans.append(active); active = None
+                if prefix == "E":
+                    spans.append((cat, c_start, c_end))
+    if active is not None:
+        spans.append(active)
+
+    return [
+        {"label": cat, "start": s, "end": e, "text": text[s:e].strip()}
+        for cat, s, e in spans
+        if text[s:e].strip()
+    ]
+```
+
+### Test
+
+#### Korean: name + phone + email
+```python
+>>> extract_pii("김민수의 전화번호는 010-1234-5678이고 이메일은 minsu@example.com입니다.")
+[
+  {"label": "private_person", "start": 0, "end": 3, "text": "김민수"},
+  {"label": "private_phone",  "start": 12, "end": 25, "text": "010-1234-5678"},
+  {"label": "private_email",  "start": 33, "end": 50, "text": "minsu@example.com"},
+]
+```
+
+#### Korean: address + name
+```python
+>>> extract_pii("서울특별시 강남구 테헤란로 123에 사는 박지영씨에게 연락주세요.")
+[
+  {"label": "private_address", "start": 0, "end": 5, "text": "서울특별시"},
+  {"label": "private_address", "start": 6, "end": 9, "text": "강남구"},
+  {"label": "private_address", "start": 10, "end": 17, "text": "테헤란로 123"},
+  {"label": "private_person",  "start": 22, "end": 25, "text": "박지영"},
+]
+```
+
+> Note: the model follows KDPII's address convention where each toponym
+> component is its own span. Most downstream redaction systems concatenate
+> adjacent address spans.
+
+#### Korean: form-style document
+```python
+>>> extract_pii('''고객 정보
+... 이름: 이수진
+... 생년월일: 1985년 3월 12일
+... 주소: 부산광역시 해운대구 우동 1457
+... 연락처: 010-9876-5432''')
+[
+  {"label": "private_person",  ..., "text": "이수진"},
+  {"label": "private_date",    ..., "text": "1985년 3월 12일"},
+  {"label": "private_address", ..., "text": "부산광역시"},
+  {"label": "private_address", ..., "text": "해운대구"},
+  {"label": "private_address", ..., "text": "우동 1457"},
+  {"label": "private_phone",   ..., "text": "010-9876-5432"},
+]
+```
+
+#### English: account + email
+```python
+>>> extract_pii("Wire to acct 110-234-567890, contact minsu@example.com")
+[
+  {"label": "account_number", "start": 13, "end": 26, "text": "110-234-567890"},
+  {"label": "private_email",  "start": 36, "end": 53, "text": "minsu@example.com"},
+]
+```
+
+### Redaction
+
+Wrap the spans into a redactor:
+
+```python
+def redact(text: str, mask: str = "[REDACTED]") -> str:
+    spans = extract_pii(text)
+    spans.sort(key=lambda s: s["start"], reverse=True)
+    out = text
+    for s in spans:
+        out = out[: s["start"]] + f"[{s['label'].upper()}]" + out[s["end"]:]
+    return out
+
+>>> redact("김민수님의 번호는 010-1234-5678입니다.")
+"[PRIVATE_PERSON]님의 번호는 [PRIVATE_PHONE]입니다."
+```
+
+## Output Schema
+
+Each detected entity is one dict:
+
+| field | description |
+|---|---|
+| `label` | One of the 9 categories above |
+| `start` | Character offset start (inclusive) |
+| `end` | Character offset end (exclusive) |
+| `text` | The matched substring |
+
+## Training Details
+
+| | |
+|---|---|
+| **Base model** | `openai/privacy-filter` (sparse MoE, 1.5B total / 50M active params, 128 experts top-4) |
+| **Method** | LoRA r=16, alpha=32, dropout=0.05 on attention projections (`q/k/v/o_proj`); classifier head fully trainable; everything else frozen |
+| **Trainable params** | ~614k (~0.04% of the model) |
+| **Datasets** | KDPII (Korean, ~53k records, deterministic 5/5/90 test/val/train), `korean_rrn_synthetic` (train only) |
+| **Optimizer** | AdamW, lr=5e-4, cosine schedule, warmup 0.1 |
+| **Batch** | 64 per device × 2 GPUs = 128 effective |
+| **Epochs** | 10, early stopping on `eval_span_f1` (patience 3) |
+| **Sequence length** | 512 |
+| **Precision** | bf16 mixed (saved as bf16 safetensors after `merge_and_unload`) |
+| **Hardware** | 2× NVIDIA RTX A5000 (24 GB each) |
+| **Final eval span F1** | 0.848 (validation) |
+
+For full reproduction details, see [`TRAINING.md`](./TRAINING.md).
+
+## Why MoE + LoRA
+
+Full fine-tuning the privacy-filter base on KDPII consistently *hurt* the
+weakest labels (`private_person` and `private_address` stuck at F1 ≈ 0.13–0.20).
+With 128 experts and top-4 routing, Korean tokens hit a small expert subset;
+across 5–10 epochs each expert receives sparse gradient updates relative to
+its parameter count, and the optimizer drags those experts away from their
+pretrained representations faster than it teaches the new task. Net effect:
+the base's pretrained Korean capability gets corrupted before the new task is
+learned.
+
+LoRA on attention only (this model) avoids this entirely — experts, FFN,
+embeddings, and router stay exactly as the base shipped them; only attention
+re-routing and the classifier head adapt. Result: F1 0.69 / 0.78 on the
+previously-stuck labels, with every other label at or above ceiling.
+
+## Known Limitations
+
+- **`private_person` residual error** is dominated by KDPII's `PS_NICKNAME`
+  policy. ~40% of remaining person errors are online-handle-style strings
+  (e.g., `탕비실맥심킹`, `퍼터요정`) that KDPII labels as `PS_NICKNAME →
+  private_person`. Downstream redaction is unaffected; classification systems
+  may want to post-classify handles separately.
+- **Foreign names** (Western, Japanese, Arabic transliterations) detected at
+  lower rates due to limited training exposure.
+- **`private_address` boundaries** follow KDPII's split convention (each
+  toponym component is a separate span). Production redactors typically
+  concatenate adjacent address spans during post-processing.
+- Raw model output may have leading/trailing whitespace in span offsets;
+  the `extract_pii` helper above strips them via `text.strip()` on the slice.
+
+## Serving with vLLM
+
+For batched, low-latency inference:
+
+```bash
+vllm serve FrameByFrame/privacy-filter-korean \
+  --task token-classification \
+  --max-model-len 512 \
+  --dtype bfloat16 \
+  --trust-remote-code
+```
+
+## License
+
+Apache 2.0 (inherited from base
+[OpenAI Privacy Filter](https://huggingface.co/openai/privacy-filter)).
+
+## Citation
+
+If you use this model:
+
+```bibtex
+@misc{framebyframe-privacy-filter-korean-2026,
+  title  = {Privacy Filter Korean: LoRA fine-tune of OpenAI Privacy Filter for Korean PII},
+  author = {FrameByFrame},
+  year   = {2026},
+  url    = {https://huggingface.co/FrameByFrame/privacy-filter-korean}
+}
+```

checkpoint-2898/README.md

@@ -0,0 +1,206 @@
+---
+base_model: /models/privacy-filter
+library_name: peft
+tags:
+- base_model:adapter:/models/privacy-filter
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

checkpoint-2898/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/models/privacy-filter",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "score",
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "TOKEN_CLS",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2898/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9c550f545e2e026d49a13fbf4ef8d15ae33e964d04565a0db49e04ee2000b1e
+size 2415282

checkpoint-2898/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3b960d61f288020cac7e198e4c3e484dfc355f182a03c8a63cc7e9a5a8285e1
+size 4870074

checkpoint-2898/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbbbe718f2a9271a2b86b705ff923bad5924435a994518a8ca80b88cc7e63027
+size 14512

checkpoint-2898/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e454729420519392bd9e7ddb031517430e22c4c8e28d4063d8b44061c052bb7
+size 14512

checkpoint-2898/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f9f8ffecc80842d5ea3bc822eb0af09a785a6d2b9310fa1410758a5de3c0ba3
+size 1064

checkpoint-2898/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e714c627d94fd333b14f9ff32436219a4d7ac969719efe340fdc3385e1c7cd3e
+size 27868272

checkpoint-2898/tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "backend": "tokenizers",
+  "eos_token": "<|endoftext|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "TokenizersBackend"
+}

checkpoint-2898/trainer_state.json

@@ -0,0 +1,1386 @@
+{
+  "best_global_step": 2898,
+  "best_metric": 0.847424684159378,
+  "best_model_checkpoint": "/workspace/data/checkpoints/ko_pii_hf_ddp_v6_lora/checkpoint-2898",
+  "epoch": 9.0,
+  "eval_steps": 500,
+  "global_step": 2898,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.003105590062111801,
+      "grad_norm": 8.771296501159668,
+      "learning_rate": 0.0,
+      "loss": 1.4503676891326904,
+      "step": 1
+    },
+    {
+      "epoch": 0.07763975155279502,
+      "grad_norm": 8.387271881103516,
+      "learning_rate": 3.7267080745341614e-05,
+      "loss": 1.2439465522766113,
+      "step": 25
+    },
+    {
+      "epoch": 0.15527950310559005,
+      "grad_norm": 4.6111741065979,
+      "learning_rate": 7.608695652173914e-05,
+      "loss": 0.9780036926269531,
+      "step": 50
+    },
+    {
+      "epoch": 0.2329192546583851,
+      "grad_norm": 1.846106767654419,
+      "learning_rate": 0.00011490683229813665,
+      "loss": 0.5338097381591796,
+      "step": 75
+    },
+    {
+      "epoch": 0.3105590062111801,
+      "grad_norm": 2.225909471511841,
+      "learning_rate": 0.00015372670807453417,
+      "loss": 0.34235782623291017,
+      "step": 100
+    },
+    {
+      "epoch": 0.38819875776397517,
+      "grad_norm": 2.14986252784729,
+      "learning_rate": 0.0001925465838509317,
+      "loss": 0.2744290542602539,
+      "step": 125
+    },
+    {
+      "epoch": 0.4658385093167702,
+      "grad_norm": 2.1372387409210205,
+      "learning_rate": 0.0002313664596273292,
+      "loss": 0.2258339309692383,
+      "step": 150
+    },
+    {
+      "epoch": 0.5434782608695652,
+      "grad_norm": 1.5849782228469849,
+      "learning_rate": 0.0002701863354037267,
+      "loss": 0.2162160873413086,
+      "step": 175
+    },
+    {
+      "epoch": 0.6211180124223602,
+      "grad_norm": 1.6991852521896362,
+      "learning_rate": 0.00030900621118012424,
+      "loss": 0.20915340423583983,
+      "step": 200
+    },
+    {
+      "epoch": 0.6987577639751553,
+      "grad_norm": 2.723663568496704,
+      "learning_rate": 0.00034782608695652176,
+      "loss": 0.18431522369384765,
+      "step": 225
+    },
+    {
+      "epoch": 0.7763975155279503,
+      "grad_norm": 1.2743035554885864,
+      "learning_rate": 0.00038664596273291923,
+      "loss": 0.17637535095214843,
+      "step": 250
+    },
+    {
+      "epoch": 0.8540372670807453,
+      "grad_norm": 3.452497720718384,
+      "learning_rate": 0.0004254658385093168,
+      "loss": 0.16074188232421874,
+      "step": 275
+    },
+    {
+      "epoch": 0.9316770186335404,
+      "grad_norm": 2.547046661376953,
+      "learning_rate": 0.00046428571428571433,
+      "loss": 0.1531696319580078,
+      "step": 300
+    },
+    {
+      "epoch": 1.0,
+      "eval_class_account_number_f1": 0.71900826446281,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.6744186046511628,
+      "eval_class_account_number_pred_spans": 129.0,
+      "eval_class_account_number_recall": 0.7699115044247787,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8070175438596492,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.7931034482758621,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.8214285714285714,
+      "eval_class_private_address_f1": 0.08450704225352113,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.13043478260869565,
+      "eval_class_private_address_pred_spans": 23.0,
+      "eval_class_private_address_recall": 0.0625,
+      "eval_class_private_date_f1": 0.8985507246376813,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.8611111111111112,
+      "eval_class_private_date_pred_spans": 36.0,
+      "eval_class_private_date_recall": 0.9393939393939394,
+      "eval_class_private_email_f1": 0.9620253164556962,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.95,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.32388663967611336,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.47619047619047616,
+      "eval_class_private_person_pred_spans": 84.0,
+      "eval_class_private_person_recall": 0.24539877300613497,
+      "eval_class_private_phone_f1": 0.9855072463768116,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 0.9855072463768116,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 0.9855072463768116,
+      "eval_class_private_url_f1": 1.0,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 1.0,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.1431565284729004,
+      "eval_pred_spans": 437.0,
+      "eval_runtime": 5.911,
+      "eval_samples_per_second": 376.753,
+      "eval_span_f1": 0.6624869383490073,
+      "eval_span_precision": 0.7254004576659039,
+      "eval_span_recall": 0.6096153846153847,
+      "eval_steps_per_second": 3.045,
+      "eval_token_accuracy": 0.9817635387488328,
+      "step": 322
+    },
+    {
+      "epoch": 1.0093167701863355,
+      "grad_norm": 1.0337207317352295,
+      "learning_rate": 0.000499999412412136,
+      "loss": 0.1343870735168457,
+      "step": 325
+    },
+    {
+      "epoch": 1.0869565217391304,
+      "grad_norm": 1.150405764579773,
+      "learning_rate": 0.0004998929197148332,
+      "loss": 0.1398656463623047,
+      "step": 350
+    },
+    {
+      "epoch": 1.1645962732919255,
+      "grad_norm": 1.239856243133545,
+      "learning_rate": 0.0004996028956208098,
+      "loss": 0.11556500434875489,
+      "step": 375
+    },
+    {
+      "epoch": 1.2422360248447206,
+      "grad_norm": 4.1494951248168945,
+      "learning_rate": 0.0004991295531354085,
+      "loss": 0.10652177810668945,
+      "step": 400
+    },
+    {
+      "epoch": 1.3198757763975155,
+      "grad_norm": 1.820974349975586,
+      "learning_rate": 0.0004984732399003618,
+      "loss": 0.1127833080291748,
+      "step": 425
+    },
+    {
+      "epoch": 1.3975155279503104,
+      "grad_norm": 0.9540935158729553,
+      "learning_rate": 0.0004976344379384687,
+      "loss": 0.1028218650817871,
+      "step": 450
+    },
+    {
+      "epoch": 1.4751552795031055,
+      "grad_norm": 1.7447997331619263,
+      "learning_rate": 0.0004966137632995793,
+      "loss": 0.11593814849853516,
+      "step": 475
+    },
+    {
+      "epoch": 1.5527950310559007,
+      "grad_norm": 1.3501027822494507,
+      "learning_rate": 0.0004954119656081417,
+      "loss": 0.1070643424987793,
+      "step": 500
+    },
+    {
+      "epoch": 1.6304347826086958,
+      "grad_norm": 0.6298630237579346,
+      "learning_rate": 0.0004940299275126486,
+      "loss": 0.11064315795898437,
+      "step": 525
+    },
+    {
+      "epoch": 1.7080745341614907,
+      "grad_norm": 0.8390702605247498,
+      "learning_rate": 0.0004924686640373841,
+      "loss": 0.09777620315551758,
+      "step": 550
+    },
+    {
+      "epoch": 1.7857142857142856,
+      "grad_norm": 0.5763865113258362,
+      "learning_rate": 0.0004907293218369499,
+      "loss": 0.08596687316894532,
+      "step": 575
+    },
+    {
+      "epoch": 1.8633540372670807,
+      "grad_norm": 0.7198231816291809,
+      "learning_rate": 0.0004888131783541152,
+      "loss": 0.0955750274658203,
+      "step": 600
+    },
+    {
+      "epoch": 1.9409937888198758,
+      "grad_norm": 1.0241382122039795,
+      "learning_rate": 0.0004867216408816122,
+      "loss": 0.0880333137512207,
+      "step": 625
+    },
+    {
+      "epoch": 2.0,
+      "eval_class_account_number_f1": 0.9184549356223176,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.8916666666666667,
+      "eval_class_account_number_pred_spans": 120.0,
+      "eval_class_account_number_recall": 0.9469026548672567,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.9310344827586207,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.9,
+      "eval_class_personal_handle_pred_spans": 30.0,
+      "eval_class_personal_handle_recall": 0.9642857142857143,
+      "eval_class_private_address_f1": 0.5747126436781609,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6410256410256411,
+      "eval_class_private_address_pred_spans": 39.0,
+      "eval_class_private_address_recall": 0.5208333333333334,
+      "eval_class_private_date_f1": 0.927536231884058,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.8888888888888888,
+      "eval_class_private_date_pred_spans": 36.0,
+      "eval_class_private_date_recall": 0.9696969696969697,
+      "eval_class_private_email_f1": 0.9743589743589743,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.9743589743589743,
+      "eval_class_private_email_pred_spans": 39.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.5182926829268292,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.5151515151515151,
+      "eval_class_private_person_pred_spans": 165.0,
+      "eval_class_private_person_recall": 0.5214723926380368,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9130434782608695,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9130434782608695,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9130434782608695,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.08562110364437103,
+      "eval_pred_spans": 525.0,
+      "eval_runtime": 6.7931,
+      "eval_samples_per_second": 327.832,
+      "eval_span_f1": 0.7808612440191388,
+      "eval_span_precision": 0.7771428571428571,
+      "eval_span_recall": 0.7846153846153846,
+      "eval_steps_per_second": 2.65,
+      "eval_token_accuracy": 0.9886788048552755,
+      "step": 644
+    },
+    {
+      "epoch": 2.018633540372671,
+      "grad_norm": 1.2737419605255127,
+      "learning_rate": 0.00048445624552856343,
+      "loss": 0.0896837329864502,
+      "step": 650
+    },
+    {
+      "epoch": 2.0962732919254656,
+      "grad_norm": 0.5823193788528442,
+      "learning_rate": 0.00048201865609230155,
+      "loss": 0.07990738868713379,
+      "step": 675
+    },
+    {
+      "epoch": 2.1739130434782608,
+      "grad_norm": 1.4650110006332397,
+      "learning_rate": 0.0004794106628364098,
+      "loss": 0.06952488899230957,
+      "step": 700
+    },
+    {
+      "epoch": 2.251552795031056,
+      "grad_norm": 0.6727014183998108,
+      "learning_rate": 0.0004766341811758803,
+      "loss": 0.07833251953125,
+      "step": 725
+    },
+    {
+      "epoch": 2.329192546583851,
+      "grad_norm": 0.6006216406822205,
+      "learning_rate": 0.000473691250270357,
+      "loss": 0.06939034461975098,
+      "step": 750
+    },
+    {
+      "epoch": 2.406832298136646,
+      "grad_norm": 0.5175172090530396,
+      "learning_rate": 0.00047058403152649405,
+      "loss": 0.05780394077301025,
+      "step": 775
+    },
+    {
+      "epoch": 2.4844720496894412,
+      "grad_norm": 1.1351436376571655,
+      "learning_rate": 0.00046731480701053265,
+      "loss": 0.06771457672119141,
+      "step": 800
+    },
+    {
+      "epoch": 2.562111801242236,
+      "grad_norm": 0.8665878772735596,
+      "learning_rate": 0.0004638859777722596,
+      "loss": 0.0703080177307129,
+      "step": 825
+    },
+    {
+      "epoch": 2.639751552795031,
+      "grad_norm": 0.8073828220367432,
+      "learning_rate": 0.00046030006208158,
+      "loss": 0.0754665756225586,
+      "step": 850
+    },
+    {
+      "epoch": 2.717391304347826,
+      "grad_norm": 0.6520082950592041,
+      "learning_rate": 0.00045655969357899875,
+      "loss": 0.06543315410614013,
+      "step": 875
+    },
+    {
+      "epoch": 2.795031055900621,
+      "grad_norm": 1.448807954788208,
+      "learning_rate": 0.00045266761934136896,
+      "loss": 0.07107664108276367,
+      "step": 900
+    },
+    {
+      "epoch": 2.8726708074534164,
+      "grad_norm": 0.45655015110969543,
+      "learning_rate": 0.00044862669786432865,
+      "loss": 0.07224170684814453,
+      "step": 925
+    },
+    {
+      "epoch": 2.950310559006211,
+      "grad_norm": 0.4825460612773895,
+      "learning_rate": 0.0004444398969629067,
+      "loss": 0.06763762950897217,
+      "step": 950
+    },
+    {
+      "epoch": 3.0,
+      "eval_class_account_number_f1": 0.9350649350649352,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9152542372881356,
+      "eval_class_account_number_pred_spans": 118.0,
+      "eval_class_account_number_recall": 0.9557522123893806,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.912280701754386,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.896551724137931,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.9285714285714286,
+      "eval_class_private_address_f1": 0.6041666666666666,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6041666666666666,
+      "eval_class_private_address_pred_spans": 48.0,
+      "eval_class_private_address_recall": 0.6041666666666666,
+      "eval_class_private_date_f1": 0.9253731343283583,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.9117647058823529,
+      "eval_class_private_date_pred_spans": 34.0,
+      "eval_class_private_date_recall": 0.9393939393939394,
+      "eval_class_private_email_f1": 0.9487179487179487,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.9487179487179487,
+      "eval_class_private_email_pred_spans": 39.0,
+      "eval_class_private_email_recall": 0.9487179487179487,
+      "eval_class_private_person_f1": 0.6013071895424836,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6433566433566433,
+      "eval_class_private_person_pred_spans": 143.0,
+      "eval_class_private_person_recall": 0.5644171779141104,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9361702127659574,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9166666666666666,
+      "eval_class_private_url_pred_spans": 24.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07642154395580292,
+      "eval_pred_spans": 508.0,
+      "eval_runtime": 6.8956,
+      "eval_samples_per_second": 322.962,
+      "eval_span_f1": 0.8132295719844358,
+      "eval_span_precision": 0.8228346456692913,
+      "eval_span_recall": 0.8038461538461539,
+      "eval_steps_per_second": 2.61,
+      "eval_token_accuracy": 0.9904586834733894,
+      "step": 966
+    },
+    {
+      "epoch": 3.027950310559006,
+      "grad_norm": 0.3684696853160858,
+      "learning_rate": 0.00044011029159184015,
+      "loss": 0.04707308769226074,
+      "step": 975
+    },
+    {
+      "epoch": 3.1055900621118013,
+      "grad_norm": 0.47226452827453613,
+      "learning_rate": 0.00043564106158720383,
+      "loss": 0.04691335678100586,
+      "step": 1000
+    },
+    {
+      "epoch": 3.1832298136645965,
+      "grad_norm": 0.6678075194358826,
+      "learning_rate": 0.00043103548933101155,
+      "loss": 0.044693613052368165,
+      "step": 1025
+    },
+    {
+      "epoch": 3.260869565217391,
+      "grad_norm": 0.675778865814209,
+      "learning_rate": 0.00042629695734050255,
+      "loss": 0.04987412452697754,
+      "step": 1050
+    },
+    {
+      "epoch": 3.3385093167701863,
+      "grad_norm": 1.2661679983139038,
+      "learning_rate": 0.00042142894578388486,
+      "loss": 0.046450929641723634,
+      "step": 1075
+    },
+    {
+      "epoch": 3.4161490683229814,
+      "grad_norm": 0.7080872058868408,
+      "learning_rate": 0.00041643502992436013,
+      "loss": 0.050359611511230466,
+      "step": 1100
+    },
+    {
+      "epoch": 3.4937888198757765,
+      "grad_norm": 0.7029626369476318,
+      "learning_rate": 0.0004113188774943071,
+      "loss": 0.06056289672851563,
+      "step": 1125
+    },
+    {
+      "epoch": 3.571428571428571,
+      "grad_norm": 0.558189332485199,
+      "learning_rate": 0.000406084246001551,
+      "loss": 0.045447168350219724,
+      "step": 1150
+    },
+    {
+      "epoch": 3.6490683229813663,
+      "grad_norm": 0.5974974036216736,
+      "learning_rate": 0.00040073497996969933,
+      "loss": 0.045512027740478515,
+      "step": 1175
+    },
+    {
+      "epoch": 3.7267080745341614,
+      "grad_norm": 0.5707910656929016,
+      "learning_rate": 0.000395275008114569,
+      "loss": 0.05025949954986572,
+      "step": 1200
+    },
+    {
+      "epoch": 3.8043478260869565,
+      "grad_norm": 1.0855952501296997,
+      "learning_rate": 0.00038970834045878026,
+      "loss": 0.06197848796844482,
+      "step": 1225
+    },
+    {
+      "epoch": 3.8819875776397517,
+      "grad_norm": 0.3965921401977539,
+      "learning_rate": 0.0003840390653866341,
+      "loss": 0.05249831199645996,
+      "step": 1250
+    },
+    {
+      "epoch": 3.9596273291925463,
+      "grad_norm": 0.5562620162963867,
+      "learning_rate": 0.00037827134664143947,
+      "loss": 0.045794315338134765,
+      "step": 1275
+    },
+    {
+      "epoch": 4.0,
+      "eval_class_account_number_f1": 0.9691629955947135,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9649122807017544,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9734513274336283,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.7719298245614034,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.7586206896551724,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.7857142857142857,
+      "eval_class_private_address_f1": 0.6067415730337078,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6585365853658537,
+      "eval_class_private_address_pred_spans": 41.0,
+      "eval_class_private_address_recall": 0.5625,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9873417721518987,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.975,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 1.0,
+      "eval_class_private_person_f1": 0.6257668711656442,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6257668711656442,
+      "eval_class_private_person_pred_spans": 163.0,
+      "eval_class_private_person_recall": 0.6257668711656442,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9565217391304348,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9565217391304348,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07150626927614212,
+      "eval_pred_spans": 516.0,
+      "eval_runtime": 6.9918,
+      "eval_samples_per_second": 318.517,
+      "eval_span_f1": 0.8262548262548264,
+      "eval_span_precision": 0.8294573643410853,
+      "eval_span_recall": 0.823076923076923,
+      "eval_steps_per_second": 2.574,
+      "eval_token_accuracy": 0.9900501867413632,
+      "step": 1288
+    },
+    {
+      "epoch": 4.037267080745342,
+      "grad_norm": 0.3308059573173523,
+      "learning_rate": 0.00037240942026749233,
+      "loss": 0.03683102369308472,
+      "step": 1300
+    },
+    {
+      "epoch": 4.114906832298137,
+      "grad_norm": 0.9377778768539429,
+      "learning_rate": 0.0003664575914989552,
+      "loss": 0.033050749301910404,
+      "step": 1325
+    },
+    {
+      "epoch": 4.192546583850931,
+      "grad_norm": 0.5735832452774048,
+      "learning_rate": 0.00036042023159791974,
+      "loss": 0.031362690925598145,
+      "step": 1350
+    },
+    {
+      "epoch": 4.270186335403727,
+      "grad_norm": 0.631287693977356,
+      "learning_rate": 0.0003543017746439766,
+      "loss": 0.03477231979370117,
+      "step": 1375
+    },
+    {
+      "epoch": 4.3478260869565215,
+      "grad_norm": 1.0935797691345215,
+      "learning_rate": 0.0003481067142776489,
+      "loss": 0.03894762754440308,
+      "step": 1400
+    },
+    {
+      "epoch": 4.425465838509317,
+      "grad_norm": 0.6073715090751648,
+      "learning_rate": 0.0003418396004000829,
+      "loss": 0.03512312889099121,
+      "step": 1425
+    },
+    {
+      "epoch": 4.503105590062112,
+      "grad_norm": 0.6249358654022217,
+      "learning_rate": 0.0003355050358314172,
+      "loss": 0.04299846649169922,
+      "step": 1450
+    },
+    {
+      "epoch": 4.580745341614907,
+      "grad_norm": 0.7248182892799377,
+      "learning_rate": 0.0003291076729302869,
+      "loss": 0.03905656576156616,
+      "step": 1475
+    },
+    {
+      "epoch": 4.658385093167702,
+      "grad_norm": 0.9138495922088623,
+      "learning_rate": 0.00032265221017694474,
+      "loss": 0.039488759040832516,
+      "step": 1500
+    },
+    {
+      "epoch": 4.736024844720497,
+      "grad_norm": 0.6601109504699707,
+      "learning_rate": 0.00031614338872250835,
+      "loss": 0.03172698974609375,
+      "step": 1525
+    },
+    {
+      "epoch": 4.813664596273292,
+      "grad_norm": 1.141296625137329,
+      "learning_rate": 0.0003095859889068687,
+      "loss": 0.038228404521942136,
+      "step": 1550
+    },
+    {
+      "epoch": 4.891304347826087,
+      "grad_norm": 1.3895987272262573,
+      "learning_rate": 0.0003029848267478158,
+      "loss": 0.03626953125,
+      "step": 1575
+    },
+    {
+      "epoch": 4.9689440993788825,
+      "grad_norm": 0.5478597283363342,
+      "learning_rate": 0.00029634475040396285,
+      "loss": 0.03073249340057373,
+      "step": 1600
+    },
+    {
+      "epoch": 5.0,
+      "eval_class_account_number_f1": 0.9691629955947135,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9649122807017544,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9734513274336283,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8771929824561403,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.8620689655172413,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.6046511627906976,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6842105263157895,
+      "eval_class_private_address_pred_spans": 38.0,
+      "eval_class_private_address_recall": 0.5416666666666666,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9873417721518987,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.975,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 1.0,
+      "eval_class_private_person_f1": 0.6027397260273972,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.5445544554455446,
+      "eval_class_private_person_pred_spans": 202.0,
+      "eval_class_private_person_recall": 0.6748466257668712,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9565217391304348,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9565217391304348,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07776352018117905,
+      "eval_pred_spans": 552.0,
+      "eval_runtime": 7.1537,
+      "eval_samples_per_second": 311.307,
+      "eval_span_f1": 0.8171641791044777,
+      "eval_span_precision": 0.7934782608695652,
+      "eval_span_recall": 0.8423076923076923,
+      "eval_steps_per_second": 2.516,
+      "eval_token_accuracy": 0.9894082633053222,
+      "step": 1610
+    },
+    {
+      "epoch": 5.046583850931677,
+      "grad_norm": 0.8306529521942139,
+      "learning_rate": 0.00028967063661406285,
+      "loss": 0.02931797981262207,
+      "step": 1625
+    },
+    {
+      "epoch": 5.124223602484472,
+      "grad_norm": 0.3057140111923218,
+      "learning_rate": 0.00028296738711533655,
+      "loss": 0.02403106212615967,
+      "step": 1650
+    },
+    {
+      "epoch": 5.201863354037267,
+      "grad_norm": 0.6968570351600647,
+      "learning_rate": 0.0002762399250434392,
+      "loss": 0.024544227123260497,
+      "step": 1675
+    },
+    {
+      "epoch": 5.279503105590062,
+      "grad_norm": 0.8571075797080994,
+      "learning_rate": 0.0002694931913167124,
+      "loss": 0.021933317184448242,
+      "step": 1700
+    },
+    {
+      "epoch": 5.357142857142857,
+      "grad_norm": 0.5366156101226807,
+      "learning_rate": 0.00026273214100737516,
+      "loss": 0.031152501106262206,
+      "step": 1725
+    },
+    {
+      "epoch": 5.434782608695652,
+      "grad_norm": 0.5963289141654968,
+      "learning_rate": 0.0002559617397023193,
+      "loss": 0.02120798110961914,
+      "step": 1750
+    },
+    {
+      "epoch": 5.512422360248447,
+      "grad_norm": 0.36561551690101624,
+      "learning_rate": 0.00024918695985618394,
+      "loss": 0.023069217205047607,
+      "step": 1775
+    },
+    {
+      "epoch": 5.590062111801243,
+      "grad_norm": 0.834469199180603,
+      "learning_rate": 0.00024241277713938443,
+      "loss": 0.018797587156295776,
+      "step": 1800
+    },
+    {
+      "epoch": 5.667701863354037,
+      "grad_norm": 0.3487168848514557,
+      "learning_rate": 0.0002356441667837804,
+      "loss": 0.027416987419128416,
+      "step": 1825
+    },
+    {
+      "epoch": 5.745341614906832,
+      "grad_norm": 0.49782446026802063,
+      "learning_rate": 0.0002288860999286656,
+      "loss": 0.029009580612182617,
+      "step": 1850
+    },
+    {
+      "epoch": 5.8229813664596275,
+      "grad_norm": 0.5541417598724365,
+      "learning_rate": 0.00022214353996976349,
+      "loss": 0.02573819875717163,
+      "step": 1875
+    },
+    {
+      "epoch": 5.900621118012422,
+      "grad_norm": 0.9394005537033081,
+      "learning_rate": 0.00021542143891391025,
+      "loss": 0.024653725624084473,
+      "step": 1900
+    },
+    {
+      "epoch": 5.978260869565218,
+      "grad_norm": 0.9744410514831543,
+      "learning_rate": 0.0002087247337421019,
+      "loss": 0.028091182708740236,
+      "step": 1925
+    },
+    {
+      "epoch": 6.0,
+      "eval_class_account_number_f1": 0.9603524229074888,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.956140350877193,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9646017699115044,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.9090909090909091,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.9259259259259259,
+      "eval_class_personal_handle_pred_spans": 27.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.6813186813186812,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7209302325581395,
+      "eval_class_private_address_pred_spans": 43.0,
+      "eval_class_private_address_recall": 0.6458333333333334,
+      "eval_class_private_date_f1": 0.9393939393939394,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.9393939393939394,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 0.9393939393939394,
+      "eval_class_private_email_f1": 0.9620253164556962,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.95,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6514657980456026,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6944444444444444,
+      "eval_class_private_person_pred_spans": 144.0,
+      "eval_class_private_person_recall": 0.6134969325153374,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9565217391304348,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9565217391304348,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07920802384614944,
+      "eval_pred_spans": 497.0,
+      "eval_runtime": 6.6716,
+      "eval_samples_per_second": 333.804,
+      "eval_span_f1": 0.8436578171091444,
+      "eval_span_precision": 0.8631790744466801,
+      "eval_span_recall": 0.825,
+      "eval_steps_per_second": 2.698,
+      "eval_token_accuracy": 0.9906921101774043,
+      "step": 1932
+    },
+    {
+      "epoch": 6.055900621118012,
+      "grad_norm": 0.5681946277618408,
+      "learning_rate": 0.00020205834278357742,
+      "loss": 0.01689502239227295,
+      "step": 1950
+    },
+    {
+      "epoch": 6.133540372670807,
+      "grad_norm": 0.3505108058452606,
+      "learning_rate": 0.00019542716210359964,
+      "loss": 0.015278645753860474,
+      "step": 1975
+    },
+    {
+      "epoch": 6.211180124223603,
+      "grad_norm": 0.3547140955924988,
+      "learning_rate": 0.00018883606190758903,
+      "loss": 0.015209052562713623,
+      "step": 2000
+    },
+    {
+      "epoch": 6.288819875776397,
+      "grad_norm": 0.3973138928413391,
+      "learning_rate": 0.00018228988296424876,
+      "loss": 0.01439091444015503,
+      "step": 2025
+    },
+    {
+      "epoch": 6.366459627329193,
+      "grad_norm": 0.5407560467720032,
+      "learning_rate": 0.00017579343305030985,
+      "loss": 0.014540103673934936,
+      "step": 2050
+    },
+    {
+      "epoch": 6.444099378881988,
+      "grad_norm": 0.3447468876838684,
+      "learning_rate": 0.00016935148341950694,
+      "loss": 0.017405078411102295,
+      "step": 2075
+    },
+    {
+      "epoch": 6.521739130434782,
+      "grad_norm": 0.30423322319984436,
+      "learning_rate": 0.0001629687652983776,
+      "loss": 0.015654172897338867,
+      "step": 2100
+    },
+    {
+      "epoch": 6.599378881987578,
+      "grad_norm": 0.47501835227012634,
+      "learning_rate": 0.0001566499664114601,
+      "loss": 0.017501153945922852,
+      "step": 2125
+    },
+    {
+      "epoch": 6.6770186335403725,
+      "grad_norm": 0.5283868312835693,
+      "learning_rate": 0.00015039972753843963,
+      "loss": 0.014991183280944825,
+      "step": 2150
+    },
+    {
+      "epoch": 6.754658385093168,
+      "grad_norm": 0.9889101982116699,
+      "learning_rate": 0.00014422263910577376,
+      "loss": 0.021359992027282716,
+      "step": 2175
+    },
+    {
+      "epoch": 6.832298136645963,
+      "grad_norm": 0.6929994225502014,
+      "learning_rate": 0.0001381232378152987,
+      "loss": 0.01291258454322815,
+      "step": 2200
+    },
+    {
+      "epoch": 6.909937888198757,
+      "grad_norm": 0.7863985896110535,
+      "learning_rate": 0.00013210600331229317,
+      "loss": 0.018135499954223634,
+      "step": 2225
+    },
+    {
+      "epoch": 6.987577639751553,
+      "grad_norm": 0.45059671998023987,
+      "learning_rate": 0.000126175354895447,
+      "loss": 0.022398817539215087,
+      "step": 2250
+    },
+    {
+      "epoch": 7.0,
+      "eval_class_account_number_f1": 0.9779735682819383,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9736842105263158,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9823008849557522,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.9090909090909091,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.9259259259259259,
+      "eval_class_personal_handle_pred_spans": 27.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.7333333333333334,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7857142857142857,
+      "eval_class_private_address_pred_spans": 42.0,
+      "eval_class_private_address_recall": 0.6875,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9500000000000001,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.926829268292683,
+      "eval_class_private_email_pred_spans": 41.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6198083067092651,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6466666666666666,
+      "eval_class_private_person_pred_spans": 150.0,
+      "eval_class_private_person_recall": 0.5950920245398773,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9583333333333334,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.92,
+      "eval_class_private_url_pred_spans": 25.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.08392658829689026,
+      "eval_pred_spans": 505.0,
+      "eval_runtime": 6.6789,
+      "eval_samples_per_second": 333.44,
+      "eval_span_f1": 0.8448780487804879,
+      "eval_span_precision": 0.8574257425742574,
+      "eval_span_recall": 0.8326923076923077,
+      "eval_steps_per_second": 2.695,
+      "eval_token_accuracy": 0.9905753968253969,
+      "step": 2254
+    },
+    {
+      "epoch": 7.065217391304348,
+      "grad_norm": 0.24804139137268066,
+      "learning_rate": 0.00012033564827115123,
+      "loss": 0.013271838426589966,
+      "step": 2275
+    },
+    {
+      "epoch": 7.142857142857143,
+      "grad_norm": 0.34166398644447327,
+      "learning_rate": 0.0001145911723544914,
+      "loss": 0.009375990629196166,
+      "step": 2300
+    },
+    {
+      "epoch": 7.220496894409938,
+      "grad_norm": 0.8205583691596985,
+      "learning_rate": 0.00010894614611929701,
+      "loss": 0.014302024841308594,
+      "step": 2325
+    },
+    {
+      "epoch": 7.298136645962733,
+      "grad_norm": 0.10090164840221405,
+      "learning_rate": 0.00010340471549955708,
+      "loss": 0.013667027950286865,
+      "step": 2350
+    },
+    {
+      "epoch": 7.375776397515528,
+      "grad_norm": 0.38398560881614685,
+      "learning_rate": 9.797095034447984e-05,
+      "loss": 0.008717143535614013,
+      "step": 2375
+    },
+    {
+      "epoch": 7.453416149068323,
+      "grad_norm": 0.4515543580055237,
+      "learning_rate": 9.264884142943237e-05,
+      "loss": 0.009974555373191833,
+      "step": 2400
+    },
+    {
+      "epoch": 7.5310559006211175,
+      "grad_norm": 0.18586605787277222,
+      "learning_rate": 8.744229752495425e-05,
+      "loss": 0.011132798194885253,
+      "step": 2425
+    },
+    {
+      "epoch": 7.608695652173913,
+      "grad_norm": 0.8527100682258606,
+      "learning_rate": 8.235514252600023e-05,
+      "loss": 0.013178889751434325,
+      "step": 2450
+    },
+    {
+      "epoch": 7.686335403726708,
+      "grad_norm": 0.08932485431432724,
+      "learning_rate": 7.739111264351752e-05,
+      "loss": 0.008569929599761963,
+      "step": 2475
+    },
+    {
+      "epoch": 7.763975155279503,
+      "grad_norm": 0.7609089612960815,
+      "learning_rate": 7.255385366042274e-05,
+      "loss": 0.01057699203491211,
+      "step": 2500
+    },
+    {
+      "epoch": 7.841614906832298,
+      "grad_norm": 0.30278390645980835,
+      "learning_rate": 6.784691825399267e-05,
+      "loss": 0.007927331924438476,
+      "step": 2525
+    },
+    {
+      "epoch": 7.919254658385093,
+      "grad_norm": 0.3382692039012909,
+      "learning_rate": 6.327376338663493e-05,
+      "loss": 0.012843213081359862,
+      "step": 2550
+    },
+    {
+      "epoch": 7.996894409937888,
+      "grad_norm": 1.0424972772598267,
+      "learning_rate": 5.883774776695652e-05,
+      "loss": 0.010109683275222778,
+      "step": 2575
+    },
+    {
+      "epoch": 8.0,
+      "eval_class_account_number_f1": 0.9779735682819383,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9736842105263158,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9823008849557522,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8214285714285714,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.8214285714285714,
+      "eval_class_personal_handle_pred_spans": 28.0,
+      "eval_class_personal_handle_recall": 0.8214285714285714,
+      "eval_class_private_address_f1": 0.6888888888888889,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7380952380952381,
+      "eval_class_private_address_pred_spans": 42.0,
+      "eval_class_private_address_recall": 0.6458333333333334,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9500000000000001,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.926829268292683,
+      "eval_class_private_email_pred_spans": 41.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6230529595015576,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6329113924050633,
+      "eval_class_private_person_pred_spans": 158.0,
+      "eval_class_private_person_recall": 0.6134969325153374,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9583333333333334,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.92,
+      "eval_class_private_url_pred_spans": 25.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.09201034158468246,
+      "eval_pred_spans": 514.0,
+      "eval_runtime": 6.9909,
+      "eval_samples_per_second": 318.559,
+      "eval_span_f1": 0.8355899419729208,
+      "eval_span_precision": 0.8404669260700389,
+      "eval_span_recall": 0.8307692307692308,
+      "eval_steps_per_second": 2.575,
+      "eval_token_accuracy": 0.9901960784313726,
+      "step": 2576
+    },
+    {
+      "epoch": 8.074534161490684,
+      "grad_norm": 0.1411687433719635,
+      "learning_rate": 5.454212938299255e-05,
+      "loss": 0.009849590063095093,
+      "step": 2600
+    },
+    {
+      "epoch": 8.152173913043478,
+      "grad_norm": 0.3560449182987213,
+      "learning_rate": 5.0390063109409914e-05,
+      "loss": 0.007954022884368896,
+      "step": 2625
+    },
+    {
+      "epoch": 8.229813664596273,
+      "grad_norm": 0.36226364970207214,
+      "learning_rate": 4.6384598390439706e-05,
+      "loss": 0.005342819690704346,
+      "step": 2650
+    },
+    {
+      "epoch": 8.307453416149068,
+      "grad_norm": 0.5297871232032776,
+      "learning_rate": 4.252867700024374e-05,
+      "loss": 0.00969659686088562,
+      "step": 2675
+    },
+    {
+      "epoch": 8.385093167701863,
+      "grad_norm": 0.5881589651107788,
+      "learning_rate": 3.882513088235676e-05,
+      "loss": 0.006519293785095215,
+      "step": 2700
+    },
+    {
+      "epoch": 8.462732919254659,
+      "grad_norm": 0.50627201795578,
+      "learning_rate": 3.5276680069793705e-05,
+      "loss": 0.00985595166683197,
+      "step": 2725
+    },
+    {
+      "epoch": 8.540372670807454,
+      "grad_norm": 0.17046724259853363,
+      "learning_rate": 3.18859306873483e-05,
+      "loss": 0.004720044732093811,
+      "step": 2750
+    },
+    {
+      "epoch": 8.618012422360248,
+      "grad_norm": 0.294168084859848,
+      "learning_rate": 2.8655373037550058e-05,
+      "loss": 0.011722609996795655,
+      "step": 2775
+    },
+    {
+      "epoch": 8.695652173913043,
+      "grad_norm": 0.7373964190483093,
+      "learning_rate": 2.5587379771686403e-05,
+      "loss": 0.006650465726852417,
+      "step": 2800
+    },
+    {
+      "epoch": 8.773291925465838,
+      "grad_norm": 0.4546540081501007,
+      "learning_rate": 2.2684204147231934e-05,
+      "loss": 0.01016199827194214,
+      "step": 2825
+    },
+    {
+      "epoch": 8.850931677018634,
+      "grad_norm": 0.9172267317771912,
+      "learning_rate": 1.9947978372965804e-05,
+      "loss": 0.009012922048568725,
+      "step": 2850
+    },
+    {
+      "epoch": 8.928571428571429,
+      "grad_norm": 0.29897835850715637,
+      "learning_rate": 1.7380712042992184e-05,
+      "loss": 0.005773799419403076,
+      "step": 2875
+    },
+    {
+      "epoch": 9.0,
+      "eval_class_account_number_f1": 0.9779735682819383,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9736842105263158,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9823008849557522,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8928571428571429,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.8928571428571429,
+      "eval_class_personal_handle_pred_spans": 28.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.7032967032967034,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7441860465116279,
+      "eval_class_private_address_pred_spans": 43.0,
+      "eval_class_private_address_recall": 0.6666666666666666,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9500000000000001,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.926829268292683,
+      "eval_class_private_email_pred_spans": 41.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6412698412698412,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6644736842105263,
+      "eval_class_private_person_pred_spans": 152.0,
+      "eval_class_private_person_recall": 0.6196319018404908,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9583333333333334,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.92,
+      "eval_class_private_url_pred_spans": 25.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.09796658158302307,
+      "eval_pred_spans": 509.0,
+      "eval_runtime": 7.2699,
+      "eval_samples_per_second": 306.33,
+      "eval_span_f1": 0.847424684159378,
+      "eval_span_precision": 0.8565815324165029,
+      "eval_span_recall": 0.8384615384615385,
+      "eval_steps_per_second": 2.476,
+      "eval_token_accuracy": 0.9907504668534081,
+      "step": 2898
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 3220,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 3,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1622112111991194e+17,
+  "train_batch_size": 64,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2898/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:195eed197af2b70c2fd47223db6e41715897a271a1bcbac65be2af77ec79752c
+size 4920

checkpoint-3220/README.md

@@ -0,0 +1,206 @@
+---
+base_model: /models/privacy-filter
+library_name: peft
+tags:
+- base_model:adapter:/models/privacy-filter
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

checkpoint-3220/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/models/privacy-filter",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "score",
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "TOKEN_CLS",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-3220/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d74f50cb3cf9daf901621a6d3c261536df2696ad33a7948d017f3c739d6fcc79
+size 2415282

checkpoint-3220/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c42b314fd5effda59213150b36df174d95e11e36e3a5d057a9b1c27bc534d33
+size 4870074

checkpoint-3220/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6228fc6b2df3522794015282d9d6ec31c5d64a50d73c37e13a7098d41fb3e4f1
+size 14512

checkpoint-3220/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fcf4f46cfe440c3d4b8b4742b0f1543f19b637bf35f17acdcd47cd790f627e2
+size 14512

checkpoint-3220/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8e680af0a1e89e60865481bc7e212dab8ba6721c1e3cc66fb1e617e0b05d8da
+size 1064

checkpoint-3220/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e714c627d94fd333b14f9ff32436219a4d7ac969719efe340fdc3385e1c7cd3e
+size 27868272

checkpoint-3220/tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "backend": "tokenizers",
+  "eos_token": "<|endoftext|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "TokenizersBackend"
+}

checkpoint-3220/trainer_state.json

@@ -0,0 +1,1536 @@
+{
+  "best_global_step": 3220,
+  "best_metric": 0.8477206595538312,
+  "best_model_checkpoint": "/workspace/data/checkpoints/ko_pii_hf_ddp_v6_lora/checkpoint-3220",
+  "epoch": 10.0,
+  "eval_steps": 500,
+  "global_step": 3220,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.003105590062111801,
+      "grad_norm": 8.771296501159668,
+      "learning_rate": 0.0,
+      "loss": 1.4503676891326904,
+      "step": 1
+    },
+    {
+      "epoch": 0.07763975155279502,
+      "grad_norm": 8.387271881103516,
+      "learning_rate": 3.7267080745341614e-05,
+      "loss": 1.2439465522766113,
+      "step": 25
+    },
+    {
+      "epoch": 0.15527950310559005,
+      "grad_norm": 4.6111741065979,
+      "learning_rate": 7.608695652173914e-05,
+      "loss": 0.9780036926269531,
+      "step": 50
+    },
+    {
+      "epoch": 0.2329192546583851,
+      "grad_norm": 1.846106767654419,
+      "learning_rate": 0.00011490683229813665,
+      "loss": 0.5338097381591796,
+      "step": 75
+    },
+    {
+      "epoch": 0.3105590062111801,
+      "grad_norm": 2.225909471511841,
+      "learning_rate": 0.00015372670807453417,
+      "loss": 0.34235782623291017,
+      "step": 100
+    },
+    {
+      "epoch": 0.38819875776397517,
+      "grad_norm": 2.14986252784729,
+      "learning_rate": 0.0001925465838509317,
+      "loss": 0.2744290542602539,
+      "step": 125
+    },
+    {
+      "epoch": 0.4658385093167702,
+      "grad_norm": 2.1372387409210205,
+      "learning_rate": 0.0002313664596273292,
+      "loss": 0.2258339309692383,
+      "step": 150
+    },
+    {
+      "epoch": 0.5434782608695652,
+      "grad_norm": 1.5849782228469849,
+      "learning_rate": 0.0002701863354037267,
+      "loss": 0.2162160873413086,
+      "step": 175
+    },
+    {
+      "epoch": 0.6211180124223602,
+      "grad_norm": 1.6991852521896362,
+      "learning_rate": 0.00030900621118012424,
+      "loss": 0.20915340423583983,
+      "step": 200
+    },
+    {
+      "epoch": 0.6987577639751553,
+      "grad_norm": 2.723663568496704,
+      "learning_rate": 0.00034782608695652176,
+      "loss": 0.18431522369384765,
+      "step": 225
+    },
+    {
+      "epoch": 0.7763975155279503,
+      "grad_norm": 1.2743035554885864,
+      "learning_rate": 0.00038664596273291923,
+      "loss": 0.17637535095214843,
+      "step": 250
+    },
+    {
+      "epoch": 0.8540372670807453,
+      "grad_norm": 3.452497720718384,
+      "learning_rate": 0.0004254658385093168,
+      "loss": 0.16074188232421874,
+      "step": 275
+    },
+    {
+      "epoch": 0.9316770186335404,
+      "grad_norm": 2.547046661376953,
+      "learning_rate": 0.00046428571428571433,
+      "loss": 0.1531696319580078,
+      "step": 300
+    },
+    {
+      "epoch": 1.0,
+      "eval_class_account_number_f1": 0.71900826446281,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.6744186046511628,
+      "eval_class_account_number_pred_spans": 129.0,
+      "eval_class_account_number_recall": 0.7699115044247787,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8070175438596492,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.7931034482758621,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.8214285714285714,
+      "eval_class_private_address_f1": 0.08450704225352113,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.13043478260869565,
+      "eval_class_private_address_pred_spans": 23.0,
+      "eval_class_private_address_recall": 0.0625,
+      "eval_class_private_date_f1": 0.8985507246376813,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.8611111111111112,
+      "eval_class_private_date_pred_spans": 36.0,
+      "eval_class_private_date_recall": 0.9393939393939394,
+      "eval_class_private_email_f1": 0.9620253164556962,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.95,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.32388663967611336,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.47619047619047616,
+      "eval_class_private_person_pred_spans": 84.0,
+      "eval_class_private_person_recall": 0.24539877300613497,
+      "eval_class_private_phone_f1": 0.9855072463768116,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 0.9855072463768116,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 0.9855072463768116,
+      "eval_class_private_url_f1": 1.0,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 1.0,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.1431565284729004,
+      "eval_pred_spans": 437.0,
+      "eval_runtime": 5.911,
+      "eval_samples_per_second": 376.753,
+      "eval_span_f1": 0.6624869383490073,
+      "eval_span_precision": 0.7254004576659039,
+      "eval_span_recall": 0.6096153846153847,
+      "eval_steps_per_second": 3.045,
+      "eval_token_accuracy": 0.9817635387488328,
+      "step": 322
+    },
+    {
+      "epoch": 1.0093167701863355,
+      "grad_norm": 1.0337207317352295,
+      "learning_rate": 0.000499999412412136,
+      "loss": 0.1343870735168457,
+      "step": 325
+    },
+    {
+      "epoch": 1.0869565217391304,
+      "grad_norm": 1.150405764579773,
+      "learning_rate": 0.0004998929197148332,
+      "loss": 0.1398656463623047,
+      "step": 350
+    },
+    {
+      "epoch": 1.1645962732919255,
+      "grad_norm": 1.239856243133545,
+      "learning_rate": 0.0004996028956208098,
+      "loss": 0.11556500434875489,
+      "step": 375
+    },
+    {
+      "epoch": 1.2422360248447206,
+      "grad_norm": 4.1494951248168945,
+      "learning_rate": 0.0004991295531354085,
+      "loss": 0.10652177810668945,
+      "step": 400
+    },
+    {
+      "epoch": 1.3198757763975155,
+      "grad_norm": 1.820974349975586,
+      "learning_rate": 0.0004984732399003618,
+      "loss": 0.1127833080291748,
+      "step": 425
+    },
+    {
+      "epoch": 1.3975155279503104,
+      "grad_norm": 0.9540935158729553,
+      "learning_rate": 0.0004976344379384687,
+      "loss": 0.1028218650817871,
+      "step": 450
+    },
+    {
+      "epoch": 1.4751552795031055,
+      "grad_norm": 1.7447997331619263,
+      "learning_rate": 0.0004966137632995793,
+      "loss": 0.11593814849853516,
+      "step": 475
+    },
+    {
+      "epoch": 1.5527950310559007,
+      "grad_norm": 1.3501027822494507,
+      "learning_rate": 0.0004954119656081417,
+      "loss": 0.1070643424987793,
+      "step": 500
+    },
+    {
+      "epoch": 1.6304347826086958,
+      "grad_norm": 0.6298630237579346,
+      "learning_rate": 0.0004940299275126486,
+      "loss": 0.11064315795898437,
+      "step": 525
+    },
+    {
+      "epoch": 1.7080745341614907,
+      "grad_norm": 0.8390702605247498,
+      "learning_rate": 0.0004924686640373841,
+      "loss": 0.09777620315551758,
+      "step": 550
+    },
+    {
+      "epoch": 1.7857142857142856,
+      "grad_norm": 0.5763865113258362,
+      "learning_rate": 0.0004907293218369499,
+      "loss": 0.08596687316894532,
+      "step": 575
+    },
+    {
+      "epoch": 1.8633540372670807,
+      "grad_norm": 0.7198231816291809,
+      "learning_rate": 0.0004888131783541152,
+      "loss": 0.0955750274658203,
+      "step": 600
+    },
+    {
+      "epoch": 1.9409937888198758,
+      "grad_norm": 1.0241382122039795,
+      "learning_rate": 0.0004867216408816122,
+      "loss": 0.0880333137512207,
+      "step": 625
+    },
+    {
+      "epoch": 2.0,
+      "eval_class_account_number_f1": 0.9184549356223176,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.8916666666666667,
+      "eval_class_account_number_pred_spans": 120.0,
+      "eval_class_account_number_recall": 0.9469026548672567,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.9310344827586207,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.9,
+      "eval_class_personal_handle_pred_spans": 30.0,
+      "eval_class_personal_handle_recall": 0.9642857142857143,
+      "eval_class_private_address_f1": 0.5747126436781609,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6410256410256411,
+      "eval_class_private_address_pred_spans": 39.0,
+      "eval_class_private_address_recall": 0.5208333333333334,
+      "eval_class_private_date_f1": 0.927536231884058,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.8888888888888888,
+      "eval_class_private_date_pred_spans": 36.0,
+      "eval_class_private_date_recall": 0.9696969696969697,
+      "eval_class_private_email_f1": 0.9743589743589743,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.9743589743589743,
+      "eval_class_private_email_pred_spans": 39.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.5182926829268292,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.5151515151515151,
+      "eval_class_private_person_pred_spans": 165.0,
+      "eval_class_private_person_recall": 0.5214723926380368,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9130434782608695,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9130434782608695,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9130434782608695,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.08562110364437103,
+      "eval_pred_spans": 525.0,
+      "eval_runtime": 6.7931,
+      "eval_samples_per_second": 327.832,
+      "eval_span_f1": 0.7808612440191388,
+      "eval_span_precision": 0.7771428571428571,
+      "eval_span_recall": 0.7846153846153846,
+      "eval_steps_per_second": 2.65,
+      "eval_token_accuracy": 0.9886788048552755,
+      "step": 644
+    },
+    {
+      "epoch": 2.018633540372671,
+      "grad_norm": 1.2737419605255127,
+      "learning_rate": 0.00048445624552856343,
+      "loss": 0.0896837329864502,
+      "step": 650
+    },
+    {
+      "epoch": 2.0962732919254656,
+      "grad_norm": 0.5823193788528442,
+      "learning_rate": 0.00048201865609230155,
+      "loss": 0.07990738868713379,
+      "step": 675
+    },
+    {
+      "epoch": 2.1739130434782608,
+      "grad_norm": 1.4650110006332397,
+      "learning_rate": 0.0004794106628364098,
+      "loss": 0.06952488899230957,
+      "step": 700
+    },
+    {
+      "epoch": 2.251552795031056,
+      "grad_norm": 0.6727014183998108,
+      "learning_rate": 0.0004766341811758803,
+      "loss": 0.07833251953125,
+      "step": 725
+    },
+    {
+      "epoch": 2.329192546583851,
+      "grad_norm": 0.6006216406822205,
+      "learning_rate": 0.000473691250270357,
+      "loss": 0.06939034461975098,
+      "step": 750
+    },
+    {
+      "epoch": 2.406832298136646,
+      "grad_norm": 0.5175172090530396,
+      "learning_rate": 0.00047058403152649405,
+      "loss": 0.05780394077301025,
+      "step": 775
+    },
+    {
+      "epoch": 2.4844720496894412,
+      "grad_norm": 1.1351436376571655,
+      "learning_rate": 0.00046731480701053265,
+      "loss": 0.06771457672119141,
+      "step": 800
+    },
+    {
+      "epoch": 2.562111801242236,
+      "grad_norm": 0.8665878772735596,
+      "learning_rate": 0.0004638859777722596,
+      "loss": 0.0703080177307129,
+      "step": 825
+    },
+    {
+      "epoch": 2.639751552795031,
+      "grad_norm": 0.8073828220367432,
+      "learning_rate": 0.00046030006208158,
+      "loss": 0.0754665756225586,
+      "step": 850
+    },
+    {
+      "epoch": 2.717391304347826,
+      "grad_norm": 0.6520082950592041,
+      "learning_rate": 0.00045655969357899875,
+      "loss": 0.06543315410614013,
+      "step": 875
+    },
+    {
+      "epoch": 2.795031055900621,
+      "grad_norm": 1.448807954788208,
+      "learning_rate": 0.00045266761934136896,
+      "loss": 0.07107664108276367,
+      "step": 900
+    },
+    {
+      "epoch": 2.8726708074534164,
+      "grad_norm": 0.45655015110969543,
+      "learning_rate": 0.00044862669786432865,
+      "loss": 0.07224170684814453,
+      "step": 925
+    },
+    {
+      "epoch": 2.950310559006211,
+      "grad_norm": 0.4825460612773895,
+      "learning_rate": 0.0004444398969629067,
+      "loss": 0.06763762950897217,
+      "step": 950
+    },
+    {
+      "epoch": 3.0,
+      "eval_class_account_number_f1": 0.9350649350649352,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9152542372881356,
+      "eval_class_account_number_pred_spans": 118.0,
+      "eval_class_account_number_recall": 0.9557522123893806,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.912280701754386,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.896551724137931,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.9285714285714286,
+      "eval_class_private_address_f1": 0.6041666666666666,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6041666666666666,
+      "eval_class_private_address_pred_spans": 48.0,
+      "eval_class_private_address_recall": 0.6041666666666666,
+      "eval_class_private_date_f1": 0.9253731343283583,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.9117647058823529,
+      "eval_class_private_date_pred_spans": 34.0,
+      "eval_class_private_date_recall": 0.9393939393939394,
+      "eval_class_private_email_f1": 0.9487179487179487,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.9487179487179487,
+      "eval_class_private_email_pred_spans": 39.0,
+      "eval_class_private_email_recall": 0.9487179487179487,
+      "eval_class_private_person_f1": 0.6013071895424836,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6433566433566433,
+      "eval_class_private_person_pred_spans": 143.0,
+      "eval_class_private_person_recall": 0.5644171779141104,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9361702127659574,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9166666666666666,
+      "eval_class_private_url_pred_spans": 24.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07642154395580292,
+      "eval_pred_spans": 508.0,
+      "eval_runtime": 6.8956,
+      "eval_samples_per_second": 322.962,
+      "eval_span_f1": 0.8132295719844358,
+      "eval_span_precision": 0.8228346456692913,
+      "eval_span_recall": 0.8038461538461539,
+      "eval_steps_per_second": 2.61,
+      "eval_token_accuracy": 0.9904586834733894,
+      "step": 966
+    },
+    {
+      "epoch": 3.027950310559006,
+      "grad_norm": 0.3684696853160858,
+      "learning_rate": 0.00044011029159184015,
+      "loss": 0.04707308769226074,
+      "step": 975
+    },
+    {
+      "epoch": 3.1055900621118013,
+      "grad_norm": 0.47226452827453613,
+      "learning_rate": 0.00043564106158720383,
+      "loss": 0.04691335678100586,
+      "step": 1000
+    },
+    {
+      "epoch": 3.1832298136645965,
+      "grad_norm": 0.6678075194358826,
+      "learning_rate": 0.00043103548933101155,
+      "loss": 0.044693613052368165,
+      "step": 1025
+    },
+    {
+      "epoch": 3.260869565217391,
+      "grad_norm": 0.675778865814209,
+      "learning_rate": 0.00042629695734050255,
+      "loss": 0.04987412452697754,
+      "step": 1050
+    },
+    {
+      "epoch": 3.3385093167701863,
+      "grad_norm": 1.2661679983139038,
+      "learning_rate": 0.00042142894578388486,
+      "loss": 0.046450929641723634,
+      "step": 1075
+    },
+    {
+      "epoch": 3.4161490683229814,
+      "grad_norm": 0.7080872058868408,
+      "learning_rate": 0.00041643502992436013,
+      "loss": 0.050359611511230466,
+      "step": 1100
+    },
+    {
+      "epoch": 3.4937888198757765,
+      "grad_norm": 0.7029626369476318,
+      "learning_rate": 0.0004113188774943071,
+      "loss": 0.06056289672851563,
+      "step": 1125
+    },
+    {
+      "epoch": 3.571428571428571,
+      "grad_norm": 0.558189332485199,
+      "learning_rate": 0.000406084246001551,
+      "loss": 0.045447168350219724,
+      "step": 1150
+    },
+    {
+      "epoch": 3.6490683229813663,
+      "grad_norm": 0.5974974036216736,
+      "learning_rate": 0.00040073497996969933,
+      "loss": 0.045512027740478515,
+      "step": 1175
+    },
+    {
+      "epoch": 3.7267080745341614,
+      "grad_norm": 0.5707910656929016,
+      "learning_rate": 0.000395275008114569,
+      "loss": 0.05025949954986572,
+      "step": 1200
+    },
+    {
+      "epoch": 3.8043478260869565,
+      "grad_norm": 1.0855952501296997,
+      "learning_rate": 0.00038970834045878026,
+      "loss": 0.06197848796844482,
+      "step": 1225
+    },
+    {
+      "epoch": 3.8819875776397517,
+      "grad_norm": 0.3965921401977539,
+      "learning_rate": 0.0003840390653866341,
+      "loss": 0.05249831199645996,
+      "step": 1250
+    },
+    {
+      "epoch": 3.9596273291925463,
+      "grad_norm": 0.5562620162963867,
+      "learning_rate": 0.00037827134664143947,
+      "loss": 0.045794315338134765,
+      "step": 1275
+    },
+    {
+      "epoch": 4.0,
+      "eval_class_account_number_f1": 0.9691629955947135,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9649122807017544,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9734513274336283,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.7719298245614034,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.7586206896551724,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.7857142857142857,
+      "eval_class_private_address_f1": 0.6067415730337078,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6585365853658537,
+      "eval_class_private_address_pred_spans": 41.0,
+      "eval_class_private_address_recall": 0.5625,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9873417721518987,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.975,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 1.0,
+      "eval_class_private_person_f1": 0.6257668711656442,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6257668711656442,
+      "eval_class_private_person_pred_spans": 163.0,
+      "eval_class_private_person_recall": 0.6257668711656442,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9565217391304348,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9565217391304348,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07150626927614212,
+      "eval_pred_spans": 516.0,
+      "eval_runtime": 6.9918,
+      "eval_samples_per_second": 318.517,
+      "eval_span_f1": 0.8262548262548264,
+      "eval_span_precision": 0.8294573643410853,
+      "eval_span_recall": 0.823076923076923,
+      "eval_steps_per_second": 2.574,
+      "eval_token_accuracy": 0.9900501867413632,
+      "step": 1288
+    },
+    {
+      "epoch": 4.037267080745342,
+      "grad_norm": 0.3308059573173523,
+      "learning_rate": 0.00037240942026749233,
+      "loss": 0.03683102369308472,
+      "step": 1300
+    },
+    {
+      "epoch": 4.114906832298137,
+      "grad_norm": 0.9377778768539429,
+      "learning_rate": 0.0003664575914989552,
+      "loss": 0.033050749301910404,
+      "step": 1325
+    },
+    {
+      "epoch": 4.192546583850931,
+      "grad_norm": 0.5735832452774048,
+      "learning_rate": 0.00036042023159791974,
+      "loss": 0.031362690925598145,
+      "step": 1350
+    },
+    {
+      "epoch": 4.270186335403727,
+      "grad_norm": 0.631287693977356,
+      "learning_rate": 0.0003543017746439766,
+      "loss": 0.03477231979370117,
+      "step": 1375
+    },
+    {
+      "epoch": 4.3478260869565215,
+      "grad_norm": 1.0935797691345215,
+      "learning_rate": 0.0003481067142776489,
+      "loss": 0.03894762754440308,
+      "step": 1400
+    },
+    {
+      "epoch": 4.425465838509317,
+      "grad_norm": 0.6073715090751648,
+      "learning_rate": 0.0003418396004000829,
+      "loss": 0.03512312889099121,
+      "step": 1425
+    },
+    {
+      "epoch": 4.503105590062112,
+      "grad_norm": 0.6249358654022217,
+      "learning_rate": 0.0003355050358314172,
+      "loss": 0.04299846649169922,
+      "step": 1450
+    },
+    {
+      "epoch": 4.580745341614907,
+      "grad_norm": 0.7248182892799377,
+      "learning_rate": 0.0003291076729302869,
+      "loss": 0.03905656576156616,
+      "step": 1475
+    },
+    {
+      "epoch": 4.658385093167702,
+      "grad_norm": 0.9138495922088623,
+      "learning_rate": 0.00032265221017694474,
+      "loss": 0.039488759040832516,
+      "step": 1500
+    },
+    {
+      "epoch": 4.736024844720497,
+      "grad_norm": 0.6601109504699707,
+      "learning_rate": 0.00031614338872250835,
+      "loss": 0.03172698974609375,
+      "step": 1525
+    },
+    {
+      "epoch": 4.813664596273292,
+      "grad_norm": 1.141296625137329,
+      "learning_rate": 0.0003095859889068687,
+      "loss": 0.038228404521942136,
+      "step": 1550
+    },
+    {
+      "epoch": 4.891304347826087,
+      "grad_norm": 1.3895987272262573,
+      "learning_rate": 0.0003029848267478158,
+      "loss": 0.03626953125,
+      "step": 1575
+    },
+    {
+      "epoch": 4.9689440993788825,
+      "grad_norm": 0.5478597283363342,
+      "learning_rate": 0.00029634475040396285,
+      "loss": 0.03073249340057373,
+      "step": 1600
+    },
+    {
+      "epoch": 5.0,
+      "eval_class_account_number_f1": 0.9691629955947135,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9649122807017544,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9734513274336283,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8771929824561403,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.8620689655172413,
+      "eval_class_personal_handle_pred_spans": 29.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.6046511627906976,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.6842105263157895,
+      "eval_class_private_address_pred_spans": 38.0,
+      "eval_class_private_address_recall": 0.5416666666666666,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9873417721518987,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.975,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 1.0,
+      "eval_class_private_person_f1": 0.6027397260273972,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.5445544554455446,
+      "eval_class_private_person_pred_spans": 202.0,
+      "eval_class_private_person_recall": 0.6748466257668712,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9565217391304348,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9565217391304348,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07776352018117905,
+      "eval_pred_spans": 552.0,
+      "eval_runtime": 7.1537,
+      "eval_samples_per_second": 311.307,
+      "eval_span_f1": 0.8171641791044777,
+      "eval_span_precision": 0.7934782608695652,
+      "eval_span_recall": 0.8423076923076923,
+      "eval_steps_per_second": 2.516,
+      "eval_token_accuracy": 0.9894082633053222,
+      "step": 1610
+    },
+    {
+      "epoch": 5.046583850931677,
+      "grad_norm": 0.8306529521942139,
+      "learning_rate": 0.00028967063661406285,
+      "loss": 0.02931797981262207,
+      "step": 1625
+    },
+    {
+      "epoch": 5.124223602484472,
+      "grad_norm": 0.3057140111923218,
+      "learning_rate": 0.00028296738711533655,
+      "loss": 0.02403106212615967,
+      "step": 1650
+    },
+    {
+      "epoch": 5.201863354037267,
+      "grad_norm": 0.6968570351600647,
+      "learning_rate": 0.0002762399250434392,
+      "loss": 0.024544227123260497,
+      "step": 1675
+    },
+    {
+      "epoch": 5.279503105590062,
+      "grad_norm": 0.8571075797080994,
+      "learning_rate": 0.0002694931913167124,
+      "loss": 0.021933317184448242,
+      "step": 1700
+    },
+    {
+      "epoch": 5.357142857142857,
+      "grad_norm": 0.5366156101226807,
+      "learning_rate": 0.00026273214100737516,
+      "loss": 0.031152501106262206,
+      "step": 1725
+    },
+    {
+      "epoch": 5.434782608695652,
+      "grad_norm": 0.5963289141654968,
+      "learning_rate": 0.0002559617397023193,
+      "loss": 0.02120798110961914,
+      "step": 1750
+    },
+    {
+      "epoch": 5.512422360248447,
+      "grad_norm": 0.36561551690101624,
+      "learning_rate": 0.00024918695985618394,
+      "loss": 0.023069217205047607,
+      "step": 1775
+    },
+    {
+      "epoch": 5.590062111801243,
+      "grad_norm": 0.834469199180603,
+      "learning_rate": 0.00024241277713938443,
+      "loss": 0.018797587156295776,
+      "step": 1800
+    },
+    {
+      "epoch": 5.667701863354037,
+      "grad_norm": 0.3487168848514557,
+      "learning_rate": 0.0002356441667837804,
+      "loss": 0.027416987419128416,
+      "step": 1825
+    },
+    {
+      "epoch": 5.745341614906832,
+      "grad_norm": 0.49782446026802063,
+      "learning_rate": 0.0002288860999286656,
+      "loss": 0.029009580612182617,
+      "step": 1850
+    },
+    {
+      "epoch": 5.8229813664596275,
+      "grad_norm": 0.5541417598724365,
+      "learning_rate": 0.00022214353996976349,
+      "loss": 0.02573819875717163,
+      "step": 1875
+    },
+    {
+      "epoch": 5.900621118012422,
+      "grad_norm": 0.9394005537033081,
+      "learning_rate": 0.00021542143891391025,
+      "loss": 0.024653725624084473,
+      "step": 1900
+    },
+    {
+      "epoch": 5.978260869565218,
+      "grad_norm": 0.9744410514831543,
+      "learning_rate": 0.0002087247337421019,
+      "loss": 0.028091182708740236,
+      "step": 1925
+    },
+    {
+      "epoch": 6.0,
+      "eval_class_account_number_f1": 0.9603524229074888,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.956140350877193,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9646017699115044,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.9090909090909091,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.9259259259259259,
+      "eval_class_personal_handle_pred_spans": 27.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.6813186813186812,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7209302325581395,
+      "eval_class_private_address_pred_spans": 43.0,
+      "eval_class_private_address_recall": 0.6458333333333334,
+      "eval_class_private_date_f1": 0.9393939393939394,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 0.9393939393939394,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 0.9393939393939394,
+      "eval_class_private_email_f1": 0.9620253164556962,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.95,
+      "eval_class_private_email_pred_spans": 40.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6514657980456026,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6944444444444444,
+      "eval_class_private_person_pred_spans": 144.0,
+      "eval_class_private_person_recall": 0.6134969325153374,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9565217391304348,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.9565217391304348,
+      "eval_class_private_url_pred_spans": 23.0,
+      "eval_class_private_url_recall": 0.9565217391304348,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.07920802384614944,
+      "eval_pred_spans": 497.0,
+      "eval_runtime": 6.6716,
+      "eval_samples_per_second": 333.804,
+      "eval_span_f1": 0.8436578171091444,
+      "eval_span_precision": 0.8631790744466801,
+      "eval_span_recall": 0.825,
+      "eval_steps_per_second": 2.698,
+      "eval_token_accuracy": 0.9906921101774043,
+      "step": 1932
+    },
+    {
+      "epoch": 6.055900621118012,
+      "grad_norm": 0.5681946277618408,
+      "learning_rate": 0.00020205834278357742,
+      "loss": 0.01689502239227295,
+      "step": 1950
+    },
+    {
+      "epoch": 6.133540372670807,
+      "grad_norm": 0.3505108058452606,
+      "learning_rate": 0.00019542716210359964,
+      "loss": 0.015278645753860474,
+      "step": 1975
+    },
+    {
+      "epoch": 6.211180124223603,
+      "grad_norm": 0.3547140955924988,
+      "learning_rate": 0.00018883606190758903,
+      "loss": 0.015209052562713623,
+      "step": 2000
+    },
+    {
+      "epoch": 6.288819875776397,
+      "grad_norm": 0.3973138928413391,
+      "learning_rate": 0.00018228988296424876,
+      "loss": 0.01439091444015503,
+      "step": 2025
+    },
+    {
+      "epoch": 6.366459627329193,
+      "grad_norm": 0.5407560467720032,
+      "learning_rate": 0.00017579343305030985,
+      "loss": 0.014540103673934936,
+      "step": 2050
+    },
+    {
+      "epoch": 6.444099378881988,
+      "grad_norm": 0.3447468876838684,
+      "learning_rate": 0.00016935148341950694,
+      "loss": 0.017405078411102295,
+      "step": 2075
+    },
+    {
+      "epoch": 6.521739130434782,
+      "grad_norm": 0.30423322319984436,
+      "learning_rate": 0.0001629687652983776,
+      "loss": 0.015654172897338867,
+      "step": 2100
+    },
+    {
+      "epoch": 6.599378881987578,
+      "grad_norm": 0.47501835227012634,
+      "learning_rate": 0.0001566499664114601,
+      "loss": 0.017501153945922852,
+      "step": 2125
+    },
+    {
+      "epoch": 6.6770186335403725,
+      "grad_norm": 0.5283868312835693,
+      "learning_rate": 0.00015039972753843963,
+      "loss": 0.014991183280944825,
+      "step": 2150
+    },
+    {
+      "epoch": 6.754658385093168,
+      "grad_norm": 0.9889101982116699,
+      "learning_rate": 0.00014422263910577376,
+      "loss": 0.021359992027282716,
+      "step": 2175
+    },
+    {
+      "epoch": 6.832298136645963,
+      "grad_norm": 0.6929994225502014,
+      "learning_rate": 0.0001381232378152987,
+      "loss": 0.01291258454322815,
+      "step": 2200
+    },
+    {
+      "epoch": 6.909937888198757,
+      "grad_norm": 0.7863985896110535,
+      "learning_rate": 0.00013210600331229317,
+      "loss": 0.018135499954223634,
+      "step": 2225
+    },
+    {
+      "epoch": 6.987577639751553,
+      "grad_norm": 0.45059671998023987,
+      "learning_rate": 0.000126175354895447,
+      "loss": 0.022398817539215087,
+      "step": 2250
+    },
+    {
+      "epoch": 7.0,
+      "eval_class_account_number_f1": 0.9779735682819383,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9736842105263158,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9823008849557522,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.9090909090909091,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.9259259259259259,
+      "eval_class_personal_handle_pred_spans": 27.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.7333333333333334,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7857142857142857,
+      "eval_class_private_address_pred_spans": 42.0,
+      "eval_class_private_address_recall": 0.6875,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9500000000000001,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.926829268292683,
+      "eval_class_private_email_pred_spans": 41.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6198083067092651,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6466666666666666,
+      "eval_class_private_person_pred_spans": 150.0,
+      "eval_class_private_person_recall": 0.5950920245398773,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9583333333333334,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.92,
+      "eval_class_private_url_pred_spans": 25.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.08392658829689026,
+      "eval_pred_spans": 505.0,
+      "eval_runtime": 6.6789,
+      "eval_samples_per_second": 333.44,
+      "eval_span_f1": 0.8448780487804879,
+      "eval_span_precision": 0.8574257425742574,
+      "eval_span_recall": 0.8326923076923077,
+      "eval_steps_per_second": 2.695,
+      "eval_token_accuracy": 0.9905753968253969,
+      "step": 2254
+    },
+    {
+      "epoch": 7.065217391304348,
+      "grad_norm": 0.24804139137268066,
+      "learning_rate": 0.00012033564827115123,
+      "loss": 0.013271838426589966,
+      "step": 2275
+    },
+    {
+      "epoch": 7.142857142857143,
+      "grad_norm": 0.34166398644447327,
+      "learning_rate": 0.0001145911723544914,
+      "loss": 0.009375990629196166,
+      "step": 2300
+    },
+    {
+      "epoch": 7.220496894409938,
+      "grad_norm": 0.8205583691596985,
+      "learning_rate": 0.00010894614611929701,
+      "loss": 0.014302024841308594,
+      "step": 2325
+    },
+    {
+      "epoch": 7.298136645962733,
+      "grad_norm": 0.10090164840221405,
+      "learning_rate": 0.00010340471549955708,
+      "loss": 0.013667027950286865,
+      "step": 2350
+    },
+    {
+      "epoch": 7.375776397515528,
+      "grad_norm": 0.38398560881614685,
+      "learning_rate": 9.797095034447984e-05,
+      "loss": 0.008717143535614013,
+      "step": 2375
+    },
+    {
+      "epoch": 7.453416149068323,
+      "grad_norm": 0.4515543580055237,
+      "learning_rate": 9.264884142943237e-05,
+      "loss": 0.009974555373191833,
+      "step": 2400
+    },
+    {
+      "epoch": 7.5310559006211175,
+      "grad_norm": 0.18586605787277222,
+      "learning_rate": 8.744229752495425e-05,
+      "loss": 0.011132798194885253,
+      "step": 2425
+    },
+    {
+      "epoch": 7.608695652173913,
+      "grad_norm": 0.8527100682258606,
+      "learning_rate": 8.235514252600023e-05,
+      "loss": 0.013178889751434325,
+      "step": 2450
+    },
+    {
+      "epoch": 7.686335403726708,
+      "grad_norm": 0.08932485431432724,
+      "learning_rate": 7.739111264351752e-05,
+      "loss": 0.008569929599761963,
+      "step": 2475
+    },
+    {
+      "epoch": 7.763975155279503,
+      "grad_norm": 0.7609089612960815,
+      "learning_rate": 7.255385366042274e-05,
+      "loss": 0.01057699203491211,
+      "step": 2500
+    },
+    {
+      "epoch": 7.841614906832298,
+      "grad_norm": 0.30278390645980835,
+      "learning_rate": 6.784691825399267e-05,
+      "loss": 0.007927331924438476,
+      "step": 2525
+    },
+    {
+      "epoch": 7.919254658385093,
+      "grad_norm": 0.3382692039012909,
+      "learning_rate": 6.327376338663493e-05,
+      "loss": 0.012843213081359862,
+      "step": 2550
+    },
+    {
+      "epoch": 7.996894409937888,
+      "grad_norm": 1.0424972772598267,
+      "learning_rate": 5.883774776695652e-05,
+      "loss": 0.010109683275222778,
+      "step": 2575
+    },
+    {
+      "epoch": 8.0,
+      "eval_class_account_number_f1": 0.9779735682819383,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9736842105263158,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9823008849557522,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8214285714285714,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.8214285714285714,
+      "eval_class_personal_handle_pred_spans": 28.0,
+      "eval_class_personal_handle_recall": 0.8214285714285714,
+      "eval_class_private_address_f1": 0.6888888888888889,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7380952380952381,
+      "eval_class_private_address_pred_spans": 42.0,
+      "eval_class_private_address_recall": 0.6458333333333334,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9500000000000001,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.926829268292683,
+      "eval_class_private_email_pred_spans": 41.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6230529595015576,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6329113924050633,
+      "eval_class_private_person_pred_spans": 158.0,
+      "eval_class_private_person_recall": 0.6134969325153374,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9583333333333334,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.92,
+      "eval_class_private_url_pred_spans": 25.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.09201034158468246,
+      "eval_pred_spans": 514.0,
+      "eval_runtime": 6.9909,
+      "eval_samples_per_second": 318.559,
+      "eval_span_f1": 0.8355899419729208,
+      "eval_span_precision": 0.8404669260700389,
+      "eval_span_recall": 0.8307692307692308,
+      "eval_steps_per_second": 2.575,
+      "eval_token_accuracy": 0.9901960784313726,
+      "step": 2576
+    },
+    {
+      "epoch": 8.074534161490684,
+      "grad_norm": 0.1411687433719635,
+      "learning_rate": 5.454212938299255e-05,
+      "loss": 0.009849590063095093,
+      "step": 2600
+    },
+    {
+      "epoch": 8.152173913043478,
+      "grad_norm": 0.3560449182987213,
+      "learning_rate": 5.0390063109409914e-05,
+      "loss": 0.007954022884368896,
+      "step": 2625
+    },
+    {
+      "epoch": 8.229813664596273,
+      "grad_norm": 0.36226364970207214,
+      "learning_rate": 4.6384598390439706e-05,
+      "loss": 0.005342819690704346,
+      "step": 2650
+    },
+    {
+      "epoch": 8.307453416149068,
+      "grad_norm": 0.5297871232032776,
+      "learning_rate": 4.252867700024374e-05,
+      "loss": 0.00969659686088562,
+      "step": 2675
+    },
+    {
+      "epoch": 8.385093167701863,
+      "grad_norm": 0.5881589651107788,
+      "learning_rate": 3.882513088235676e-05,
+      "loss": 0.006519293785095215,
+      "step": 2700
+    },
+    {
+      "epoch": 8.462732919254659,
+      "grad_norm": 0.50627201795578,
+      "learning_rate": 3.5276680069793705e-05,
+      "loss": 0.00985595166683197,
+      "step": 2725
+    },
+    {
+      "epoch": 8.540372670807454,
+      "grad_norm": 0.17046724259853363,
+      "learning_rate": 3.18859306873483e-05,
+      "loss": 0.004720044732093811,
+      "step": 2750
+    },
+    {
+      "epoch": 8.618012422360248,
+      "grad_norm": 0.294168084859848,
+      "learning_rate": 2.8655373037550058e-05,
+      "loss": 0.011722609996795655,
+      "step": 2775
+    },
+    {
+      "epoch": 8.695652173913043,
+      "grad_norm": 0.7373964190483093,
+      "learning_rate": 2.5587379771686403e-05,
+      "loss": 0.006650465726852417,
+      "step": 2800
+    },
+    {
+      "epoch": 8.773291925465838,
+      "grad_norm": 0.4546540081501007,
+      "learning_rate": 2.2684204147231934e-05,
+      "loss": 0.01016199827194214,
+      "step": 2825
+    },
+    {
+      "epoch": 8.850931677018634,
+      "grad_norm": 0.9172267317771912,
+      "learning_rate": 1.9947978372965804e-05,
+      "loss": 0.009012922048568725,
+      "step": 2850
+    },
+    {
+      "epoch": 8.928571428571429,
+      "grad_norm": 0.29897835850715637,
+      "learning_rate": 1.7380712042992184e-05,
+      "loss": 0.005773799419403076,
+      "step": 2875
+    },
+    {
+      "epoch": 9.0,
+      "eval_class_account_number_f1": 0.9779735682819383,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9736842105263158,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9823008849557522,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8928571428571429,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.8928571428571429,
+      "eval_class_personal_handle_pred_spans": 28.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.7032967032967034,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7441860465116279,
+      "eval_class_private_address_pred_spans": 43.0,
+      "eval_class_private_address_recall": 0.6666666666666666,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9500000000000001,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.926829268292683,
+      "eval_class_private_email_pred_spans": 41.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6412698412698412,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6644736842105263,
+      "eval_class_private_person_pred_spans": 152.0,
+      "eval_class_private_person_recall": 0.6196319018404908,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9583333333333334,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.92,
+      "eval_class_private_url_pred_spans": 25.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.09796658158302307,
+      "eval_pred_spans": 509.0,
+      "eval_runtime": 7.2699,
+      "eval_samples_per_second": 306.33,
+      "eval_span_f1": 0.847424684159378,
+      "eval_span_precision": 0.8565815324165029,
+      "eval_span_recall": 0.8384615384615385,
+      "eval_steps_per_second": 2.476,
+      "eval_token_accuracy": 0.9907504668534081,
+      "step": 2898
+    },
+    {
+      "epoch": 9.006211180124224,
+      "grad_norm": 0.1876034289598465,
+      "learning_rate": 1.498429066081311e-05,
+      "loss": 0.008111558556556701,
+      "step": 2900
+    },
+    {
+      "epoch": 9.083850931677018,
+      "grad_norm": 0.23115915060043335,
+      "learning_rate": 1.276047425453955e-05,
+      "loss": 0.007400209903717041,
+      "step": 2925
+    },
+    {
+      "epoch": 9.161490683229813,
+      "grad_norm": 0.30637550354003906,
+      "learning_rate": 1.0710896084255684e-05,
+      "loss": 0.007166728377342224,
+      "step": 2950
+    },
+    {
+      "epoch": 9.23913043478261,
+      "grad_norm": 0.2221984565258026,
+      "learning_rate": 8.837061442487193e-06,
+      "loss": 0.006242750883102417,
+      "step": 2975
+    },
+    {
+      "epoch": 9.316770186335404,
+      "grad_norm": 0.38621774315834045,
+      "learning_rate": 7.1403465486540584e-06,
+      "loss": 0.008013453483581543,
+      "step": 3000
+    },
+    {
+      "epoch": 9.394409937888199,
+      "grad_norm": 0.2457902729511261,
+      "learning_rate": 5.6219975383197675e-06,
+      "loss": 0.008517981171607972,
+      "step": 3025
+    },
+    {
+      "epoch": 9.472049689440993,
+      "grad_norm": 0.5641500353813171,
+      "learning_rate": 4.2831295479791034e-06,
+      "loss": 0.006109429001808167,
+      "step": 3050
+    },
+    {
+      "epoch": 9.549689440993788,
+      "grad_norm": 0.18688508868217468,
+      "learning_rate": 3.1247258960573067e-06,
+      "loss": 0.005958260297775269,
+      "step": 3075
+    },
+    {
+      "epoch": 9.627329192546584,
+      "grad_norm": 0.2731750011444092,
+      "learning_rate": 2.1476373607214882e-06,
+      "loss": 0.009225964546203613,
+      "step": 3100
+    },
+    {
+      "epoch": 9.70496894409938,
+      "grad_norm": 0.12572576105594635,
+      "learning_rate": 1.3525815550349262e-06,
+      "loss": 0.0062656128406524654,
+      "step": 3125
+    },
+    {
+      "epoch": 9.782608695652174,
+      "grad_norm": 0.28346896171569824,
+      "learning_rate": 7.401423999132406e-07,
+      "loss": 0.0059302222728729245,
+      "step": 3150
+    },
+    {
+      "epoch": 9.860248447204969,
+      "grad_norm": 0.4496349096298218,
+      "learning_rate": 3.107696952694139e-07,
+      "loss": 0.005810813903808594,
+      "step": 3175
+    },
+    {
+      "epoch": 9.937888198757763,
+      "grad_norm": 0.7214242815971375,
+      "learning_rate": 6.477878966271167e-08,
+      "loss": 0.008135276436805726,
+      "step": 3200
+    },
+    {
+      "epoch": 10.0,
+      "eval_class_account_number_f1": 0.9779735682819383,
+      "eval_class_account_number_gold_spans": 113.0,
+      "eval_class_account_number_precision": 0.9736842105263158,
+      "eval_class_account_number_pred_spans": 114.0,
+      "eval_class_account_number_recall": 0.9823008849557522,
+      "eval_class_ip_address_f1": 1.0,
+      "eval_class_ip_address_gold_spans": 4.0,
+      "eval_class_ip_address_precision": 1.0,
+      "eval_class_ip_address_pred_spans": 4.0,
+      "eval_class_ip_address_recall": 1.0,
+      "eval_class_personal_handle_f1": 0.8928571428571429,
+      "eval_class_personal_handle_gold_spans": 28.0,
+      "eval_class_personal_handle_precision": 0.8928571428571429,
+      "eval_class_personal_handle_pred_spans": 28.0,
+      "eval_class_personal_handle_recall": 0.8928571428571429,
+      "eval_class_private_address_f1": 0.7032967032967034,
+      "eval_class_private_address_gold_spans": 48.0,
+      "eval_class_private_address_precision": 0.7441860465116279,
+      "eval_class_private_address_pred_spans": 43.0,
+      "eval_class_private_address_recall": 0.6666666666666666,
+      "eval_class_private_date_f1": 1.0,
+      "eval_class_private_date_gold_spans": 33.0,
+      "eval_class_private_date_precision": 1.0,
+      "eval_class_private_date_pred_spans": 33.0,
+      "eval_class_private_date_recall": 1.0,
+      "eval_class_private_email_f1": 0.9500000000000001,
+      "eval_class_private_email_gold_spans": 39.0,
+      "eval_class_private_email_precision": 0.926829268292683,
+      "eval_class_private_email_pred_spans": 41.0,
+      "eval_class_private_email_recall": 0.9743589743589743,
+      "eval_class_private_person_f1": 0.6435331230283912,
+      "eval_class_private_person_gold_spans": 163.0,
+      "eval_class_private_person_precision": 0.6623376623376623,
+      "eval_class_private_person_pred_spans": 154.0,
+      "eval_class_private_person_recall": 0.6257668711656442,
+      "eval_class_private_phone_f1": 1.0,
+      "eval_class_private_phone_gold_spans": 69.0,
+      "eval_class_private_phone_precision": 1.0,
+      "eval_class_private_phone_pred_spans": 69.0,
+      "eval_class_private_phone_recall": 1.0,
+      "eval_class_private_url_f1": 0.9583333333333334,
+      "eval_class_private_url_gold_spans": 23.0,
+      "eval_class_private_url_precision": 0.92,
+      "eval_class_private_url_pred_spans": 25.0,
+      "eval_class_private_url_recall": 1.0,
+      "eval_gold_spans": 520.0,
+      "eval_loss": 0.09789847582578659,
+      "eval_pred_spans": 511.0,
+      "eval_runtime": 7.0696,
+      "eval_samples_per_second": 315.012,
+      "eval_span_f1": 0.8477206595538312,
+      "eval_span_precision": 0.8551859099804305,
+      "eval_span_recall": 0.8403846153846154,
+      "eval_steps_per_second": 2.546,
+      "eval_token_accuracy": 0.9908088235294118,
+      "step": 3220
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 3220,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 3,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.291714856788951e+17,
+  "train_batch_size": 64,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-3220/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:195eed197af2b70c2fd47223db6e41715897a271a1bcbac65be2af77ec79752c
+size 4920

config.json

@@ -0,0 +1,128 @@
+{
+  "architectures": [
+    "OpenAIPrivacyFilterForTokenClassification"
+  ],
+  "attention_bias": true,
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "classifier_dropout": 0.0,
+  "default_n_ctx": 128000,
+  "dtype": "bfloat16",
+  "eos_token_id": 199999,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 640,
+  "id2label": {
+    "0": "O",
+    "1": "B-private_person",
+    "2": "I-private_person",
+    "3": "E-private_person",
+    "4": "S-private_person",
+    "5": "B-personal_handle",
+    "6": "I-personal_handle",
+    "7": "E-personal_handle",
+    "8": "S-personal_handle",
+    "9": "B-private_phone",
+    "10": "I-private_phone",
+    "11": "E-private_phone",
+    "12": "S-private_phone",
+    "13": "B-private_email",
+    "14": "I-private_email",
+    "15": "E-private_email",
+    "16": "S-private_email",
+    "17": "B-private_address",
+    "18": "I-private_address",
+    "19": "E-private_address",
+    "20": "S-private_address",
+    "21": "B-private_date",
+    "22": "I-private_date",
+    "23": "E-private_date",
+    "24": "S-private_date",
+    "25": "B-private_url",
+    "26": "I-private_url",
+    "27": "E-private_url",
+    "28": "S-private_url",
+    "29": "B-account_number",
+    "30": "I-account_number",
+    "31": "E-account_number",
+    "32": "S-account_number",
+    "33": "B-ip_address",
+    "34": "I-ip_address",
+    "35": "E-ip_address",
+    "36": "S-ip_address"
+  },
+  "initial_context_length": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 640,
+  "label2id": {
+    "B-account_number": 29,
+    "B-ip_address": 33,
+    "B-personal_handle": 5,
+    "B-private_address": 17,
+    "B-private_date": 21,
+    "B-private_email": 13,
+    "B-private_person": 1,
+    "B-private_phone": 9,
+    "B-private_url": 25,
+    "E-account_number": 31,
+    "E-ip_address": 35,
+    "E-personal_handle": 7,
+    "E-private_address": 19,
+    "E-private_date": 23,
+    "E-private_email": 15,
+    "E-private_person": 3,
+    "E-private_phone": 11,
+    "E-private_url": 27,
+    "I-account_number": 30,
+    "I-ip_address": 34,
+    "I-personal_handle": 6,
+    "I-private_address": 18,
+    "I-private_date": 22,
+    "I-private_email": 14,
+    "I-private_person": 2,
+    "I-private_phone": 10,
+    "I-private_url": 26,
+    "O": 0,
+    "S-account_number": 32,
+    "S-ip_address": 36,
+    "S-personal_handle": 8,
+    "S-private_address": 20,
+    "S-private_date": 24,
+    "S-private_email": 16,
+    "S-private_person": 4,
+    "S-private_phone": 12,
+    "S-private_url": 28
+  },
+  "max_position_embeddings": 131072,
+  "model_type": "openai_privacy_filter",
+  "num_attention_heads": 14,
+  "num_experts_per_tok": 4,
+  "num_hidden_layers": 8,
+  "num_key_value_heads": 2,
+  "num_local_experts": 128,
+  "output_router_logits": false,
+  "pad_token_id": 199999,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "beta_fast": 32.0,
+    "beta_slow": 1.0,
+    "factor": 32.0,
+    "original_max_position_embeddings": 4096,
+    "rope_theta": 150000.0,
+    "rope_type": "yarn",
+    "truncate": false
+  },
+  "router_aux_loss_coef": 0.001,
+  "sliding_window": 128,
+  "tie_word_embeddings": false,
+  "transformers.js_config": {
+    "use_external_data_format": {
+      "model": 1,
+      "model.onnx": 3,
+      "model_fp16.onnx": 2
+    }
+  },
+  "transformers_version": "5.7.0.dev0",
+  "use_cache": false,
+  "vocab_size": 200064
+}

label_space.json

@@ -0,0 +1,15 @@
+{
+  "category_version": "ko_pii_v4",
+  "span_class_names": [
+    "O",
+    "private_person",
+    "personal_handle",
+    "private_phone",
+    "private_email",
+    "private_address",
+    "private_date",
+    "private_url",
+    "account_number",
+    "ip_address"
+  ]
+}

log_history.json

@@ -0,0 +1,1622 @@
+[
+  {
+    "loss": 1.4503676891326904,
+    "grad_norm": 8.771296501159668,
+    "learning_rate": 0.0,
+    "epoch": 0.003105590062111801,
+    "step": 1
+  },
+  {
+    "loss": 1.2439465522766113,
+    "grad_norm": 8.387271881103516,
+    "learning_rate": 3.7267080745341614e-05,
+    "epoch": 0.07763975155279502,
+    "step": 25
+  },
+  {
+    "loss": 0.9780036926269531,
+    "grad_norm": 4.6111741065979,
+    "learning_rate": 7.608695652173914e-05,
+    "epoch": 0.15527950310559005,
+    "step": 50
+  },
+  {
+    "loss": 0.5338097381591796,
+    "grad_norm": 1.846106767654419,
+    "learning_rate": 0.00011490683229813665,
+    "epoch": 0.2329192546583851,
+    "step": 75
+  },
+  {
+    "loss": 0.34235782623291017,
+    "grad_norm": 2.225909471511841,
+    "learning_rate": 0.00015372670807453417,
+    "epoch": 0.3105590062111801,
+    "step": 100
+  },
+  {
+    "loss": 0.2744290542602539,
+    "grad_norm": 2.14986252784729,
+    "learning_rate": 0.0001925465838509317,
+    "epoch": 0.38819875776397517,
+    "step": 125
+  },
+  {
+    "loss": 0.2258339309692383,
+    "grad_norm": 2.1372387409210205,
+    "learning_rate": 0.0002313664596273292,
+    "epoch": 0.4658385093167702,
+    "step": 150
+  },
+  {
+    "loss": 0.2162160873413086,
+    "grad_norm": 1.5849782228469849,
+    "learning_rate": 0.0002701863354037267,
+    "epoch": 0.5434782608695652,
+    "step": 175
+  },
+  {
+    "loss": 0.20915340423583983,
+    "grad_norm": 1.6991852521896362,
+    "learning_rate": 0.00030900621118012424,
+    "epoch": 0.6211180124223602,
+    "step": 200
+  },
+  {
+    "loss": 0.18431522369384765,
+    "grad_norm": 2.723663568496704,
+    "learning_rate": 0.00034782608695652176,
+    "epoch": 0.6987577639751553,
+    "step": 225
+  },
+  {
+    "loss": 0.17637535095214843,
+    "grad_norm": 1.2743035554885864,
+    "learning_rate": 0.00038664596273291923,
+    "epoch": 0.7763975155279503,
+    "step": 250
+  },
+  {
+    "loss": 0.16074188232421874,
+    "grad_norm": 3.452497720718384,
+    "learning_rate": 0.0004254658385093168,
+    "epoch": 0.8540372670807453,
+    "step": 275
+  },
+  {
+    "loss": 0.1531696319580078,
+    "grad_norm": 2.547046661376953,
+    "learning_rate": 0.00046428571428571433,
+    "epoch": 0.9316770186335404,
+    "step": 300
+  },
+  {
+    "eval_loss": 0.1431565284729004,
+    "eval_token_accuracy": 0.9817635387488328,
+    "eval_span_precision": 0.7254004576659039,
+    "eval_span_recall": 0.6096153846153847,
+    "eval_span_f1": 0.6624869383490073,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 437.0,
+    "eval_class_account_number_precision": 0.6744186046511628,
+    "eval_class_account_number_recall": 0.7699115044247787,
+    "eval_class_account_number_f1": 0.71900826446281,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 129.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.7931034482758621,
+    "eval_class_personal_handle_recall": 0.8214285714285714,
+    "eval_class_personal_handle_f1": 0.8070175438596492,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 29.0,
+    "eval_class_private_address_precision": 0.13043478260869565,
+    "eval_class_private_address_recall": 0.0625,
+    "eval_class_private_address_f1": 0.08450704225352113,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 23.0,
+    "eval_class_private_date_precision": 0.8611111111111112,
+    "eval_class_private_date_recall": 0.9393939393939394,
+    "eval_class_private_date_f1": 0.8985507246376813,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 36.0,
+    "eval_class_private_email_precision": 0.95,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9620253164556962,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 40.0,
+    "eval_class_private_person_precision": 0.47619047619047616,
+    "eval_class_private_person_recall": 0.24539877300613497,
+    "eval_class_private_person_f1": 0.32388663967611336,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 84.0,
+    "eval_class_private_phone_precision": 0.9855072463768116,
+    "eval_class_private_phone_recall": 0.9855072463768116,
+    "eval_class_private_phone_f1": 0.9855072463768116,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 1.0,
+    "eval_class_private_url_recall": 1.0,
+    "eval_class_private_url_f1": 1.0,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 23.0,
+    "eval_runtime": 5.911,
+    "eval_samples_per_second": 376.753,
+    "eval_steps_per_second": 3.045,
+    "epoch": 1.0,
+    "step": 322
+  },
+  {
+    "loss": 0.1343870735168457,
+    "grad_norm": 1.0337207317352295,
+    "learning_rate": 0.000499999412412136,
+    "epoch": 1.0093167701863355,
+    "step": 325
+  },
+  {
+    "loss": 0.1398656463623047,
+    "grad_norm": 1.150405764579773,
+    "learning_rate": 0.0004998929197148332,
+    "epoch": 1.0869565217391304,
+    "step": 350
+  },
+  {
+    "loss": 0.11556500434875489,
+    "grad_norm": 1.239856243133545,
+    "learning_rate": 0.0004996028956208098,
+    "epoch": 1.1645962732919255,
+    "step": 375
+  },
+  {
+    "loss": 0.10652177810668945,
+    "grad_norm": 4.1494951248168945,
+    "learning_rate": 0.0004991295531354085,
+    "epoch": 1.2422360248447206,
+    "step": 400
+  },
+  {
+    "loss": 0.1127833080291748,
+    "grad_norm": 1.820974349975586,
+    "learning_rate": 0.0004984732399003618,
+    "epoch": 1.3198757763975155,
+    "step": 425
+  },
+  {
+    "loss": 0.1028218650817871,
+    "grad_norm": 0.9540935158729553,
+    "learning_rate": 0.0004976344379384687,
+    "epoch": 1.3975155279503104,
+    "step": 450
+  },
+  {
+    "loss": 0.11593814849853516,
+    "grad_norm": 1.7447997331619263,
+    "learning_rate": 0.0004966137632995793,
+    "epoch": 1.4751552795031055,
+    "step": 475
+  },
+  {
+    "loss": 0.1070643424987793,
+    "grad_norm": 1.3501027822494507,
+    "learning_rate": 0.0004954119656081417,
+    "epoch": 1.5527950310559007,
+    "step": 500
+  },
+  {
+    "loss": 0.11064315795898437,
+    "grad_norm": 0.6298630237579346,
+    "learning_rate": 0.0004940299275126486,
+    "epoch": 1.6304347826086958,
+    "step": 525
+  },
+  {
+    "loss": 0.09777620315551758,
+    "grad_norm": 0.8390702605247498,
+    "learning_rate": 0.0004924686640373841,
+    "epoch": 1.7080745341614907,
+    "step": 550
+  },
+  {
+    "loss": 0.08596687316894532,
+    "grad_norm": 0.5763865113258362,
+    "learning_rate": 0.0004907293218369499,
+    "epoch": 1.7857142857142856,
+    "step": 575
+  },
+  {
+    "loss": 0.0955750274658203,
+    "grad_norm": 0.7198231816291809,
+    "learning_rate": 0.0004888131783541152,
+    "epoch": 1.8633540372670807,
+    "step": 600
+  },
+  {
+    "loss": 0.0880333137512207,
+    "grad_norm": 1.0241382122039795,
+    "learning_rate": 0.0004867216408816122,
+    "epoch": 1.9409937888198758,
+    "step": 625
+  },
+  {
+    "eval_loss": 0.08562110364437103,
+    "eval_token_accuracy": 0.9886788048552755,
+    "eval_span_precision": 0.7771428571428571,
+    "eval_span_recall": 0.7846153846153846,
+    "eval_span_f1": 0.7808612440191388,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 525.0,
+    "eval_class_account_number_precision": 0.8916666666666667,
+    "eval_class_account_number_recall": 0.9469026548672567,
+    "eval_class_account_number_f1": 0.9184549356223176,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 120.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.9,
+    "eval_class_personal_handle_recall": 0.9642857142857143,
+    "eval_class_personal_handle_f1": 0.9310344827586207,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 30.0,
+    "eval_class_private_address_precision": 0.6410256410256411,
+    "eval_class_private_address_recall": 0.5208333333333334,
+    "eval_class_private_address_f1": 0.5747126436781609,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 39.0,
+    "eval_class_private_date_precision": 0.8888888888888888,
+    "eval_class_private_date_recall": 0.9696969696969697,
+    "eval_class_private_date_f1": 0.927536231884058,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 36.0,
+    "eval_class_private_email_precision": 0.9743589743589743,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9743589743589743,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 39.0,
+    "eval_class_private_person_precision": 0.5151515151515151,
+    "eval_class_private_person_recall": 0.5214723926380368,
+    "eval_class_private_person_f1": 0.5182926829268292,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 165.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.9130434782608695,
+    "eval_class_private_url_recall": 0.9130434782608695,
+    "eval_class_private_url_f1": 0.9130434782608695,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 23.0,
+    "eval_runtime": 6.7931,
+    "eval_samples_per_second": 327.832,
+    "eval_steps_per_second": 2.65,
+    "epoch": 2.0,
+    "step": 644
+  },
+  {
+    "loss": 0.0896837329864502,
+    "grad_norm": 1.2737419605255127,
+    "learning_rate": 0.00048445624552856343,
+    "epoch": 2.018633540372671,
+    "step": 650
+  },
+  {
+    "loss": 0.07990738868713379,
+    "grad_norm": 0.5823193788528442,
+    "learning_rate": 0.00048201865609230155,
+    "epoch": 2.0962732919254656,
+    "step": 675
+  },
+  {
+    "loss": 0.06952488899230957,
+    "grad_norm": 1.4650110006332397,
+    "learning_rate": 0.0004794106628364098,
+    "epoch": 2.1739130434782608,
+    "step": 700
+  },
+  {
+    "loss": 0.07833251953125,
+    "grad_norm": 0.6727014183998108,
+    "learning_rate": 0.0004766341811758803,
+    "epoch": 2.251552795031056,
+    "step": 725
+  },
+  {
+    "loss": 0.06939034461975098,
+    "grad_norm": 0.6006216406822205,
+    "learning_rate": 0.000473691250270357,
+    "epoch": 2.329192546583851,
+    "step": 750
+  },
+  {
+    "loss": 0.05780394077301025,
+    "grad_norm": 0.5175172090530396,
+    "learning_rate": 0.00047058403152649405,
+    "epoch": 2.406832298136646,
+    "step": 775
+  },
+  {
+    "loss": 0.06771457672119141,
+    "grad_norm": 1.1351436376571655,
+    "learning_rate": 0.00046731480701053265,
+    "epoch": 2.4844720496894412,
+    "step": 800
+  },
+  {
+    "loss": 0.0703080177307129,
+    "grad_norm": 0.8665878772735596,
+    "learning_rate": 0.0004638859777722596,
+    "epoch": 2.562111801242236,
+    "step": 825
+  },
+  {
+    "loss": 0.0754665756225586,
+    "grad_norm": 0.8073828220367432,
+    "learning_rate": 0.00046030006208158,
+    "epoch": 2.639751552795031,
+    "step": 850
+  },
+  {
+    "loss": 0.06543315410614013,
+    "grad_norm": 0.6520082950592041,
+    "learning_rate": 0.00045655969357899875,
+    "epoch": 2.717391304347826,
+    "step": 875
+  },
+  {
+    "loss": 0.07107664108276367,
+    "grad_norm": 1.448807954788208,
+    "learning_rate": 0.00045266761934136896,
+    "epoch": 2.795031055900621,
+    "step": 900
+  },
+  {
+    "loss": 0.07224170684814453,
+    "grad_norm": 0.45655015110969543,
+    "learning_rate": 0.00044862669786432865,
+    "epoch": 2.8726708074534164,
+    "step": 925
+  },
+  {
+    "loss": 0.06763762950897217,
+    "grad_norm": 0.4825460612773895,
+    "learning_rate": 0.0004444398969629067,
+    "epoch": 2.950310559006211,
+    "step": 950
+  },
+  {
+    "eval_loss": 0.07642154395580292,
+    "eval_token_accuracy": 0.9904586834733894,
+    "eval_span_precision": 0.8228346456692913,
+    "eval_span_recall": 0.8038461538461539,
+    "eval_span_f1": 0.8132295719844358,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 508.0,
+    "eval_class_account_number_precision": 0.9152542372881356,
+    "eval_class_account_number_recall": 0.9557522123893806,
+    "eval_class_account_number_f1": 0.9350649350649352,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 118.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.896551724137931,
+    "eval_class_personal_handle_recall": 0.9285714285714286,
+    "eval_class_personal_handle_f1": 0.912280701754386,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 29.0,
+    "eval_class_private_address_precision": 0.6041666666666666,
+    "eval_class_private_address_recall": 0.6041666666666666,
+    "eval_class_private_address_f1": 0.6041666666666666,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 48.0,
+    "eval_class_private_date_precision": 0.9117647058823529,
+    "eval_class_private_date_recall": 0.9393939393939394,
+    "eval_class_private_date_f1": 0.9253731343283583,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 34.0,
+    "eval_class_private_email_precision": 0.9487179487179487,
+    "eval_class_private_email_recall": 0.9487179487179487,
+    "eval_class_private_email_f1": 0.9487179487179487,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 39.0,
+    "eval_class_private_person_precision": 0.6433566433566433,
+    "eval_class_private_person_recall": 0.5644171779141104,
+    "eval_class_private_person_f1": 0.6013071895424836,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 143.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.9166666666666666,
+    "eval_class_private_url_recall": 0.9565217391304348,
+    "eval_class_private_url_f1": 0.9361702127659574,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 24.0,
+    "eval_runtime": 6.8956,
+    "eval_samples_per_second": 322.962,
+    "eval_steps_per_second": 2.61,
+    "epoch": 3.0,
+    "step": 966
+  },
+  {
+    "loss": 0.04707308769226074,
+    "grad_norm": 0.3684696853160858,
+    "learning_rate": 0.00044011029159184015,
+    "epoch": 3.027950310559006,
+    "step": 975
+  },
+  {
+    "loss": 0.04691335678100586,
+    "grad_norm": 0.47226452827453613,
+    "learning_rate": 0.00043564106158720383,
+    "epoch": 3.1055900621118013,
+    "step": 1000
+  },
+  {
+    "loss": 0.044693613052368165,
+    "grad_norm": 0.6678075194358826,
+    "learning_rate": 0.00043103548933101155,
+    "epoch": 3.1832298136645965,
+    "step": 1025
+  },
+  {
+    "loss": 0.04987412452697754,
+    "grad_norm": 0.675778865814209,
+    "learning_rate": 0.00042629695734050255,
+    "epoch": 3.260869565217391,
+    "step": 1050
+  },
+  {
+    "loss": 0.046450929641723634,
+    "grad_norm": 1.2661679983139038,
+    "learning_rate": 0.00042142894578388486,
+    "epoch": 3.3385093167701863,
+    "step": 1075
+  },
+  {
+    "loss": 0.050359611511230466,
+    "grad_norm": 0.7080872058868408,
+    "learning_rate": 0.00041643502992436013,
+    "epoch": 3.4161490683229814,
+    "step": 1100
+  },
+  {
+    "loss": 0.06056289672851563,
+    "grad_norm": 0.7029626369476318,
+    "learning_rate": 0.0004113188774943071,
+    "epoch": 3.4937888198757765,
+    "step": 1125
+  },
+  {
+    "loss": 0.045447168350219724,
+    "grad_norm": 0.558189332485199,
+    "learning_rate": 0.000406084246001551,
+    "epoch": 3.571428571428571,
+    "step": 1150
+  },
+  {
+    "loss": 0.045512027740478515,
+    "grad_norm": 0.5974974036216736,
+    "learning_rate": 0.00040073497996969933,
+    "epoch": 3.6490683229813663,
+    "step": 1175
+  },
+  {
+    "loss": 0.05025949954986572,
+    "grad_norm": 0.5707910656929016,
+    "learning_rate": 0.000395275008114569,
+    "epoch": 3.7267080745341614,
+    "step": 1200
+  },
+  {
+    "loss": 0.06197848796844482,
+    "grad_norm": 1.0855952501296997,
+    "learning_rate": 0.00038970834045878026,
+    "epoch": 3.8043478260869565,
+    "step": 1225
+  },
+  {
+    "loss": 0.05249831199645996,
+    "grad_norm": 0.3965921401977539,
+    "learning_rate": 0.0003840390653866341,
+    "epoch": 3.8819875776397517,
+    "step": 1250
+  },
+  {
+    "loss": 0.045794315338134765,
+    "grad_norm": 0.5562620162963867,
+    "learning_rate": 0.00037827134664143947,
+    "epoch": 3.9596273291925463,
+    "step": 1275
+  },
+  {
+    "eval_loss": 0.07150626927614212,
+    "eval_token_accuracy": 0.9900501867413632,
+    "eval_span_precision": 0.8294573643410853,
+    "eval_span_recall": 0.823076923076923,
+    "eval_span_f1": 0.8262548262548264,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 516.0,
+    "eval_class_account_number_precision": 0.9649122807017544,
+    "eval_class_account_number_recall": 0.9734513274336283,
+    "eval_class_account_number_f1": 0.9691629955947135,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.7586206896551724,
+    "eval_class_personal_handle_recall": 0.7857142857142857,
+    "eval_class_personal_handle_f1": 0.7719298245614034,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 29.0,
+    "eval_class_private_address_precision": 0.6585365853658537,
+    "eval_class_private_address_recall": 0.5625,
+    "eval_class_private_address_f1": 0.6067415730337078,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 41.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.975,
+    "eval_class_private_email_recall": 1.0,
+    "eval_class_private_email_f1": 0.9873417721518987,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 40.0,
+    "eval_class_private_person_precision": 0.6257668711656442,
+    "eval_class_private_person_recall": 0.6257668711656442,
+    "eval_class_private_person_f1": 0.6257668711656442,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 163.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.9565217391304348,
+    "eval_class_private_url_recall": 0.9565217391304348,
+    "eval_class_private_url_f1": 0.9565217391304348,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 23.0,
+    "eval_runtime": 6.9918,
+    "eval_samples_per_second": 318.517,
+    "eval_steps_per_second": 2.574,
+    "epoch": 4.0,
+    "step": 1288
+  },
+  {
+    "loss": 0.03683102369308472,
+    "grad_norm": 0.3308059573173523,
+    "learning_rate": 0.00037240942026749233,
+    "epoch": 4.037267080745342,
+    "step": 1300
+  },
+  {
+    "loss": 0.033050749301910404,
+    "grad_norm": 0.9377778768539429,
+    "learning_rate": 0.0003664575914989552,
+    "epoch": 4.114906832298137,
+    "step": 1325
+  },
+  {
+    "loss": 0.031362690925598145,
+    "grad_norm": 0.5735832452774048,
+    "learning_rate": 0.00036042023159791974,
+    "epoch": 4.192546583850931,
+    "step": 1350
+  },
+  {
+    "loss": 0.03477231979370117,
+    "grad_norm": 0.631287693977356,
+    "learning_rate": 0.0003543017746439766,
+    "epoch": 4.270186335403727,
+    "step": 1375
+  },
+  {
+    "loss": 0.03894762754440308,
+    "grad_norm": 1.0935797691345215,
+    "learning_rate": 0.0003481067142776489,
+    "epoch": 4.3478260869565215,
+    "step": 1400
+  },
+  {
+    "loss": 0.03512312889099121,
+    "grad_norm": 0.6073715090751648,
+    "learning_rate": 0.0003418396004000829,
+    "epoch": 4.425465838509317,
+    "step": 1425
+  },
+  {
+    "loss": 0.04299846649169922,
+    "grad_norm": 0.6249358654022217,
+    "learning_rate": 0.0003355050358314172,
+    "epoch": 4.503105590062112,
+    "step": 1450
+  },
+  {
+    "loss": 0.03905656576156616,
+    "grad_norm": 0.7248182892799377,
+    "learning_rate": 0.0003291076729302869,
+    "epoch": 4.580745341614907,
+    "step": 1475
+  },
+  {
+    "loss": 0.039488759040832516,
+    "grad_norm": 0.9138495922088623,
+    "learning_rate": 0.00032265221017694474,
+    "epoch": 4.658385093167702,
+    "step": 1500
+  },
+  {
+    "loss": 0.03172698974609375,
+    "grad_norm": 0.6601109504699707,
+    "learning_rate": 0.00031614338872250835,
+    "epoch": 4.736024844720497,
+    "step": 1525
+  },
+  {
+    "loss": 0.038228404521942136,
+    "grad_norm": 1.141296625137329,
+    "learning_rate": 0.0003095859889068687,
+    "epoch": 4.813664596273292,
+    "step": 1550
+  },
+  {
+    "loss": 0.03626953125,
+    "grad_norm": 1.3895987272262573,
+    "learning_rate": 0.0003029848267478158,
+    "epoch": 4.891304347826087,
+    "step": 1575
+  },
+  {
+    "loss": 0.03073249340057373,
+    "grad_norm": 0.5478597283363342,
+    "learning_rate": 0.00029634475040396285,
+    "epoch": 4.9689440993788825,
+    "step": 1600
+  },
+  {
+    "eval_loss": 0.07776352018117905,
+    "eval_token_accuracy": 0.9894082633053222,
+    "eval_span_precision": 0.7934782608695652,
+    "eval_span_recall": 0.8423076923076923,
+    "eval_span_f1": 0.8171641791044777,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 552.0,
+    "eval_class_account_number_precision": 0.9649122807017544,
+    "eval_class_account_number_recall": 0.9734513274336283,
+    "eval_class_account_number_f1": 0.9691629955947135,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.8620689655172413,
+    "eval_class_personal_handle_recall": 0.8928571428571429,
+    "eval_class_personal_handle_f1": 0.8771929824561403,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 29.0,
+    "eval_class_private_address_precision": 0.6842105263157895,
+    "eval_class_private_address_recall": 0.5416666666666666,
+    "eval_class_private_address_f1": 0.6046511627906976,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 38.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.975,
+    "eval_class_private_email_recall": 1.0,
+    "eval_class_private_email_f1": 0.9873417721518987,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 40.0,
+    "eval_class_private_person_precision": 0.5445544554455446,
+    "eval_class_private_person_recall": 0.6748466257668712,
+    "eval_class_private_person_f1": 0.6027397260273972,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 202.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.9565217391304348,
+    "eval_class_private_url_recall": 0.9565217391304348,
+    "eval_class_private_url_f1": 0.9565217391304348,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 23.0,
+    "eval_runtime": 7.1537,
+    "eval_samples_per_second": 311.307,
+    "eval_steps_per_second": 2.516,
+    "epoch": 5.0,
+    "step": 1610
+  },
+  {
+    "loss": 0.02931797981262207,
+    "grad_norm": 0.8306529521942139,
+    "learning_rate": 0.00028967063661406285,
+    "epoch": 5.046583850931677,
+    "step": 1625
+  },
+  {
+    "loss": 0.02403106212615967,
+    "grad_norm": 0.3057140111923218,
+    "learning_rate": 0.00028296738711533655,
+    "epoch": 5.124223602484472,
+    "step": 1650
+  },
+  {
+    "loss": 0.024544227123260497,
+    "grad_norm": 0.6968570351600647,
+    "learning_rate": 0.0002762399250434392,
+    "epoch": 5.201863354037267,
+    "step": 1675
+  },
+  {
+    "loss": 0.021933317184448242,
+    "grad_norm": 0.8571075797080994,
+    "learning_rate": 0.0002694931913167124,
+    "epoch": 5.279503105590062,
+    "step": 1700
+  },
+  {
+    "loss": 0.031152501106262206,
+    "grad_norm": 0.5366156101226807,
+    "learning_rate": 0.00026273214100737516,
+    "epoch": 5.357142857142857,
+    "step": 1725
+  },
+  {
+    "loss": 0.02120798110961914,
+    "grad_norm": 0.5963289141654968,
+    "learning_rate": 0.0002559617397023193,
+    "epoch": 5.434782608695652,
+    "step": 1750
+  },
+  {
+    "loss": 0.023069217205047607,
+    "grad_norm": 0.36561551690101624,
+    "learning_rate": 0.00024918695985618394,
+    "epoch": 5.512422360248447,
+    "step": 1775
+  },
+  {
+    "loss": 0.018797587156295776,
+    "grad_norm": 0.834469199180603,
+    "learning_rate": 0.00024241277713938443,
+    "epoch": 5.590062111801243,
+    "step": 1800
+  },
+  {
+    "loss": 0.027416987419128416,
+    "grad_norm": 0.3487168848514557,
+    "learning_rate": 0.0002356441667837804,
+    "epoch": 5.667701863354037,
+    "step": 1825
+  },
+  {
+    "loss": 0.029009580612182617,
+    "grad_norm": 0.49782446026802063,
+    "learning_rate": 0.0002288860999286656,
+    "epoch": 5.745341614906832,
+    "step": 1850
+  },
+  {
+    "loss": 0.02573819875717163,
+    "grad_norm": 0.5541417598724365,
+    "learning_rate": 0.00022214353996976349,
+    "epoch": 5.8229813664596275,
+    "step": 1875
+  },
+  {
+    "loss": 0.024653725624084473,
+    "grad_norm": 0.9394005537033081,
+    "learning_rate": 0.00021542143891391025,
+    "epoch": 5.900621118012422,
+    "step": 1900
+  },
+  {
+    "loss": 0.028091182708740236,
+    "grad_norm": 0.9744410514831543,
+    "learning_rate": 0.0002087247337421019,
+    "epoch": 5.978260869565218,
+    "step": 1925
+  },
+  {
+    "eval_loss": 0.07920802384614944,
+    "eval_token_accuracy": 0.9906921101774043,
+    "eval_span_precision": 0.8631790744466801,
+    "eval_span_recall": 0.825,
+    "eval_span_f1": 0.8436578171091444,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 497.0,
+    "eval_class_account_number_precision": 0.956140350877193,
+    "eval_class_account_number_recall": 0.9646017699115044,
+    "eval_class_account_number_f1": 0.9603524229074888,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.9259259259259259,
+    "eval_class_personal_handle_recall": 0.8928571428571429,
+    "eval_class_personal_handle_f1": 0.9090909090909091,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 27.0,
+    "eval_class_private_address_precision": 0.7209302325581395,
+    "eval_class_private_address_recall": 0.6458333333333334,
+    "eval_class_private_address_f1": 0.6813186813186812,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 43.0,
+    "eval_class_private_date_precision": 0.9393939393939394,
+    "eval_class_private_date_recall": 0.9393939393939394,
+    "eval_class_private_date_f1": 0.9393939393939394,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.95,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9620253164556962,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 40.0,
+    "eval_class_private_person_precision": 0.6944444444444444,
+    "eval_class_private_person_recall": 0.6134969325153374,
+    "eval_class_private_person_f1": 0.6514657980456026,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 144.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.9565217391304348,
+    "eval_class_private_url_recall": 0.9565217391304348,
+    "eval_class_private_url_f1": 0.9565217391304348,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 23.0,
+    "eval_runtime": 6.6716,
+    "eval_samples_per_second": 333.804,
+    "eval_steps_per_second": 2.698,
+    "epoch": 6.0,
+    "step": 1932
+  },
+  {
+    "loss": 0.01689502239227295,
+    "grad_norm": 0.5681946277618408,
+    "learning_rate": 0.00020205834278357742,
+    "epoch": 6.055900621118012,
+    "step": 1950
+  },
+  {
+    "loss": 0.015278645753860474,
+    "grad_norm": 0.3505108058452606,
+    "learning_rate": 0.00019542716210359964,
+    "epoch": 6.133540372670807,
+    "step": 1975
+  },
+  {
+    "loss": 0.015209052562713623,
+    "grad_norm": 0.3547140955924988,
+    "learning_rate": 0.00018883606190758903,
+    "epoch": 6.211180124223603,
+    "step": 2000
+  },
+  {
+    "loss": 0.01439091444015503,
+    "grad_norm": 0.3973138928413391,
+    "learning_rate": 0.00018228988296424876,
+    "epoch": 6.288819875776397,
+    "step": 2025
+  },
+  {
+    "loss": 0.014540103673934936,
+    "grad_norm": 0.5407560467720032,
+    "learning_rate": 0.00017579343305030985,
+    "epoch": 6.366459627329193,
+    "step": 2050
+  },
+  {
+    "loss": 0.017405078411102295,
+    "grad_norm": 0.3447468876838684,
+    "learning_rate": 0.00016935148341950694,
+    "epoch": 6.444099378881988,
+    "step": 2075
+  },
+  {
+    "loss": 0.015654172897338867,
+    "grad_norm": 0.30423322319984436,
+    "learning_rate": 0.0001629687652983776,
+    "epoch": 6.521739130434782,
+    "step": 2100
+  },
+  {
+    "loss": 0.017501153945922852,
+    "grad_norm": 0.47501835227012634,
+    "learning_rate": 0.0001566499664114601,
+    "epoch": 6.599378881987578,
+    "step": 2125
+  },
+  {
+    "loss": 0.014991183280944825,
+    "grad_norm": 0.5283868312835693,
+    "learning_rate": 0.00015039972753843963,
+    "epoch": 6.6770186335403725,
+    "step": 2150
+  },
+  {
+    "loss": 0.021359992027282716,
+    "grad_norm": 0.9889101982116699,
+    "learning_rate": 0.00014422263910577376,
+    "epoch": 6.754658385093168,
+    "step": 2175
+  },
+  {
+    "loss": 0.01291258454322815,
+    "grad_norm": 0.6929994225502014,
+    "learning_rate": 0.0001381232378152987,
+    "epoch": 6.832298136645963,
+    "step": 2200
+  },
+  {
+    "loss": 0.018135499954223634,
+    "grad_norm": 0.7863985896110535,
+    "learning_rate": 0.00013210600331229317,
+    "epoch": 6.909937888198757,
+    "step": 2225
+  },
+  {
+    "loss": 0.022398817539215087,
+    "grad_norm": 0.45059671998023987,
+    "learning_rate": 0.000126175354895447,
+    "epoch": 6.987577639751553,
+    "step": 2250
+  },
+  {
+    "eval_loss": 0.08392658829689026,
+    "eval_token_accuracy": 0.9905753968253969,
+    "eval_span_precision": 0.8574257425742574,
+    "eval_span_recall": 0.8326923076923077,
+    "eval_span_f1": 0.8448780487804879,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 505.0,
+    "eval_class_account_number_precision": 0.9736842105263158,
+    "eval_class_account_number_recall": 0.9823008849557522,
+    "eval_class_account_number_f1": 0.9779735682819383,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.9259259259259259,
+    "eval_class_personal_handle_recall": 0.8928571428571429,
+    "eval_class_personal_handle_f1": 0.9090909090909091,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 27.0,
+    "eval_class_private_address_precision": 0.7857142857142857,
+    "eval_class_private_address_recall": 0.6875,
+    "eval_class_private_address_f1": 0.7333333333333334,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 42.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.926829268292683,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9500000000000001,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 41.0,
+    "eval_class_private_person_precision": 0.6466666666666666,
+    "eval_class_private_person_recall": 0.5950920245398773,
+    "eval_class_private_person_f1": 0.6198083067092651,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 150.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.92,
+    "eval_class_private_url_recall": 1.0,
+    "eval_class_private_url_f1": 0.9583333333333334,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 25.0,
+    "eval_runtime": 6.6789,
+    "eval_samples_per_second": 333.44,
+    "eval_steps_per_second": 2.695,
+    "epoch": 7.0,
+    "step": 2254
+  },
+  {
+    "loss": 0.013271838426589966,
+    "grad_norm": 0.24804139137268066,
+    "learning_rate": 0.00012033564827115123,
+    "epoch": 7.065217391304348,
+    "step": 2275
+  },
+  {
+    "loss": 0.009375990629196166,
+    "grad_norm": 0.34166398644447327,
+    "learning_rate": 0.0001145911723544914,
+    "epoch": 7.142857142857143,
+    "step": 2300
+  },
+  {
+    "loss": 0.014302024841308594,
+    "grad_norm": 0.8205583691596985,
+    "learning_rate": 0.00010894614611929701,
+    "epoch": 7.220496894409938,
+    "step": 2325
+  },
+  {
+    "loss": 0.013667027950286865,
+    "grad_norm": 0.10090164840221405,
+    "learning_rate": 0.00010340471549955708,
+    "epoch": 7.298136645962733,
+    "step": 2350
+  },
+  {
+    "loss": 0.008717143535614013,
+    "grad_norm": 0.38398560881614685,
+    "learning_rate": 9.797095034447984e-05,
+    "epoch": 7.375776397515528,
+    "step": 2375
+  },
+  {
+    "loss": 0.009974555373191833,
+    "grad_norm": 0.4515543580055237,
+    "learning_rate": 9.264884142943237e-05,
+    "epoch": 7.453416149068323,
+    "step": 2400
+  },
+  {
+    "loss": 0.011132798194885253,
+    "grad_norm": 0.18586605787277222,
+    "learning_rate": 8.744229752495425e-05,
+    "epoch": 7.5310559006211175,
+    "step": 2425
+  },
+  {
+    "loss": 0.013178889751434325,
+    "grad_norm": 0.8527100682258606,
+    "learning_rate": 8.235514252600023e-05,
+    "epoch": 7.608695652173913,
+    "step": 2450
+  },
+  {
+    "loss": 0.008569929599761963,
+    "grad_norm": 0.08932485431432724,
+    "learning_rate": 7.739111264351752e-05,
+    "epoch": 7.686335403726708,
+    "step": 2475
+  },
+  {
+    "loss": 0.01057699203491211,
+    "grad_norm": 0.7609089612960815,
+    "learning_rate": 7.255385366042274e-05,
+    "epoch": 7.763975155279503,
+    "step": 2500
+  },
+  {
+    "loss": 0.007927331924438476,
+    "grad_norm": 0.30278390645980835,
+    "learning_rate": 6.784691825399267e-05,
+    "epoch": 7.841614906832298,
+    "step": 2525
+  },
+  {
+    "loss": 0.012843213081359862,
+    "grad_norm": 0.3382692039012909,
+    "learning_rate": 6.327376338663493e-05,
+    "epoch": 7.919254658385093,
+    "step": 2550
+  },
+  {
+    "loss": 0.010109683275222778,
+    "grad_norm": 1.0424972772598267,
+    "learning_rate": 5.883774776695652e-05,
+    "epoch": 7.996894409937888,
+    "step": 2575
+  },
+  {
+    "eval_loss": 0.09201034158468246,
+    "eval_token_accuracy": 0.9901960784313726,
+    "eval_span_precision": 0.8404669260700389,
+    "eval_span_recall": 0.8307692307692308,
+    "eval_span_f1": 0.8355899419729208,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 514.0,
+    "eval_class_account_number_precision": 0.9736842105263158,
+    "eval_class_account_number_recall": 0.9823008849557522,
+    "eval_class_account_number_f1": 0.9779735682819383,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.8214285714285714,
+    "eval_class_personal_handle_recall": 0.8214285714285714,
+    "eval_class_personal_handle_f1": 0.8214285714285714,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 28.0,
+    "eval_class_private_address_precision": 0.7380952380952381,
+    "eval_class_private_address_recall": 0.6458333333333334,
+    "eval_class_private_address_f1": 0.6888888888888889,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 42.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.926829268292683,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9500000000000001,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 41.0,
+    "eval_class_private_person_precision": 0.6329113924050633,
+    "eval_class_private_person_recall": 0.6134969325153374,
+    "eval_class_private_person_f1": 0.6230529595015576,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 158.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.92,
+    "eval_class_private_url_recall": 1.0,
+    "eval_class_private_url_f1": 0.9583333333333334,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 25.0,
+    "eval_runtime": 6.9909,
+    "eval_samples_per_second": 318.559,
+    "eval_steps_per_second": 2.575,
+    "epoch": 8.0,
+    "step": 2576
+  },
+  {
+    "loss": 0.009849590063095093,
+    "grad_norm": 0.1411687433719635,
+    "learning_rate": 5.454212938299255e-05,
+    "epoch": 8.074534161490684,
+    "step": 2600
+  },
+  {
+    "loss": 0.007954022884368896,
+    "grad_norm": 0.3560449182987213,
+    "learning_rate": 5.0390063109409914e-05,
+    "epoch": 8.152173913043478,
+    "step": 2625
+  },
+  {
+    "loss": 0.005342819690704346,
+    "grad_norm": 0.36226364970207214,
+    "learning_rate": 4.6384598390439706e-05,
+    "epoch": 8.229813664596273,
+    "step": 2650
+  },
+  {
+    "loss": 0.00969659686088562,
+    "grad_norm": 0.5297871232032776,
+    "learning_rate": 4.252867700024374e-05,
+    "epoch": 8.307453416149068,
+    "step": 2675
+  },
+  {
+    "loss": 0.006519293785095215,
+    "grad_norm": 0.5881589651107788,
+    "learning_rate": 3.882513088235676e-05,
+    "epoch": 8.385093167701863,
+    "step": 2700
+  },
+  {
+    "loss": 0.00985595166683197,
+    "grad_norm": 0.50627201795578,
+    "learning_rate": 3.5276680069793705e-05,
+    "epoch": 8.462732919254659,
+    "step": 2725
+  },
+  {
+    "loss": 0.004720044732093811,
+    "grad_norm": 0.17046724259853363,
+    "learning_rate": 3.18859306873483e-05,
+    "epoch": 8.540372670807454,
+    "step": 2750
+  },
+  {
+    "loss": 0.011722609996795655,
+    "grad_norm": 0.294168084859848,
+    "learning_rate": 2.8655373037550058e-05,
+    "epoch": 8.618012422360248,
+    "step": 2775
+  },
+  {
+    "loss": 0.006650465726852417,
+    "grad_norm": 0.7373964190483093,
+    "learning_rate": 2.5587379771686403e-05,
+    "epoch": 8.695652173913043,
+    "step": 2800
+  },
+  {
+    "loss": 0.01016199827194214,
+    "grad_norm": 0.4546540081501007,
+    "learning_rate": 2.2684204147231934e-05,
+    "epoch": 8.773291925465838,
+    "step": 2825
+  },
+  {
+    "loss": 0.009012922048568725,
+    "grad_norm": 0.9172267317771912,
+    "learning_rate": 1.9947978372965804e-05,
+    "epoch": 8.850931677018634,
+    "step": 2850
+  },
+  {
+    "loss": 0.005773799419403076,
+    "grad_norm": 0.29897835850715637,
+    "learning_rate": 1.7380712042992184e-05,
+    "epoch": 8.928571428571429,
+    "step": 2875
+  },
+  {
+    "eval_loss": 0.09796658158302307,
+    "eval_token_accuracy": 0.9907504668534081,
+    "eval_span_precision": 0.8565815324165029,
+    "eval_span_recall": 0.8384615384615385,
+    "eval_span_f1": 0.847424684159378,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 509.0,
+    "eval_class_account_number_precision": 0.9736842105263158,
+    "eval_class_account_number_recall": 0.9823008849557522,
+    "eval_class_account_number_f1": 0.9779735682819383,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.8928571428571429,
+    "eval_class_personal_handle_recall": 0.8928571428571429,
+    "eval_class_personal_handle_f1": 0.8928571428571429,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 28.0,
+    "eval_class_private_address_precision": 0.7441860465116279,
+    "eval_class_private_address_recall": 0.6666666666666666,
+    "eval_class_private_address_f1": 0.7032967032967034,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 43.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.926829268292683,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9500000000000001,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 41.0,
+    "eval_class_private_person_precision": 0.6644736842105263,
+    "eval_class_private_person_recall": 0.6196319018404908,
+    "eval_class_private_person_f1": 0.6412698412698412,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 152.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.92,
+    "eval_class_private_url_recall": 1.0,
+    "eval_class_private_url_f1": 0.9583333333333334,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 25.0,
+    "eval_runtime": 7.2699,
+    "eval_samples_per_second": 306.33,
+    "eval_steps_per_second": 2.476,
+    "epoch": 9.0,
+    "step": 2898
+  },
+  {
+    "loss": 0.008111558556556701,
+    "grad_norm": 0.1876034289598465,
+    "learning_rate": 1.498429066081311e-05,
+    "epoch": 9.006211180124224,
+    "step": 2900
+  },
+  {
+    "loss": 0.007400209903717041,
+    "grad_norm": 0.23115915060043335,
+    "learning_rate": 1.276047425453955e-05,
+    "epoch": 9.083850931677018,
+    "step": 2925
+  },
+  {
+    "loss": 0.007166728377342224,
+    "grad_norm": 0.30637550354003906,
+    "learning_rate": 1.0710896084255684e-05,
+    "epoch": 9.161490683229813,
+    "step": 2950
+  },
+  {
+    "loss": 0.006242750883102417,
+    "grad_norm": 0.2221984565258026,
+    "learning_rate": 8.837061442487193e-06,
+    "epoch": 9.23913043478261,
+    "step": 2975
+  },
+  {
+    "loss": 0.008013453483581543,
+    "grad_norm": 0.38621774315834045,
+    "learning_rate": 7.1403465486540584e-06,
+    "epoch": 9.316770186335404,
+    "step": 3000
+  },
+  {
+    "loss": 0.008517981171607972,
+    "grad_norm": 0.2457902729511261,
+    "learning_rate": 5.6219975383197675e-06,
+    "epoch": 9.394409937888199,
+    "step": 3025
+  },
+  {
+    "loss": 0.006109429001808167,
+    "grad_norm": 0.5641500353813171,
+    "learning_rate": 4.2831295479791034e-06,
+    "epoch": 9.472049689440993,
+    "step": 3050
+  },
+  {
+    "loss": 0.005958260297775269,
+    "grad_norm": 0.18688508868217468,
+    "learning_rate": 3.1247258960573067e-06,
+    "epoch": 9.549689440993788,
+    "step": 3075
+  },
+  {
+    "loss": 0.009225964546203613,
+    "grad_norm": 0.2731750011444092,
+    "learning_rate": 2.1476373607214882e-06,
+    "epoch": 9.627329192546584,
+    "step": 3100
+  },
+  {
+    "loss": 0.0062656128406524654,
+    "grad_norm": 0.12572576105594635,
+    "learning_rate": 1.3525815550349262e-06,
+    "epoch": 9.70496894409938,
+    "step": 3125
+  },
+  {
+    "loss": 0.0059302222728729245,
+    "grad_norm": 0.28346896171569824,
+    "learning_rate": 7.401423999132406e-07,
+    "epoch": 9.782608695652174,
+    "step": 3150
+  },
+  {
+    "loss": 0.005810813903808594,
+    "grad_norm": 0.4496349096298218,
+    "learning_rate": 3.107696952694139e-07,
+    "epoch": 9.860248447204969,
+    "step": 3175
+  },
+  {
+    "loss": 0.008135276436805726,
+    "grad_norm": 0.7214242815971375,
+    "learning_rate": 6.477878966271167e-08,
+    "epoch": 9.937888198757763,
+    "step": 3200
+  },
+  {
+    "eval_loss": 0.09789847582578659,
+    "eval_token_accuracy": 0.9908088235294118,
+    "eval_span_precision": 0.8551859099804305,
+    "eval_span_recall": 0.8403846153846154,
+    "eval_span_f1": 0.8477206595538312,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 511.0,
+    "eval_class_account_number_precision": 0.9736842105263158,
+    "eval_class_account_number_recall": 0.9823008849557522,
+    "eval_class_account_number_f1": 0.9779735682819383,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.8928571428571429,
+    "eval_class_personal_handle_recall": 0.8928571428571429,
+    "eval_class_personal_handle_f1": 0.8928571428571429,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 28.0,
+    "eval_class_private_address_precision": 0.7441860465116279,
+    "eval_class_private_address_recall": 0.6666666666666666,
+    "eval_class_private_address_f1": 0.7032967032967034,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 43.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.926829268292683,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9500000000000001,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 41.0,
+    "eval_class_private_person_precision": 0.6623376623376623,
+    "eval_class_private_person_recall": 0.6257668711656442,
+    "eval_class_private_person_f1": 0.6435331230283912,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 154.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.92,
+    "eval_class_private_url_recall": 1.0,
+    "eval_class_private_url_f1": 0.9583333333333334,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 25.0,
+    "eval_runtime": 7.0696,
+    "eval_samples_per_second": 315.012,
+    "eval_steps_per_second": 2.546,
+    "epoch": 10.0,
+    "step": 3220
+  },
+  {
+    "train_runtime": 2753.5271,
+    "train_samples_per_second": 149.615,
+    "train_steps_per_second": 1.169,
+    "total_flos": 1.291714856788951e+17,
+    "train_loss": 0.07032274794504509,
+    "epoch": 10.0,
+    "step": 3220
+  },
+  {
+    "eval_loss": 0.09874702990055084,
+    "eval_token_accuracy": 0.9907504668534081,
+    "eval_span_precision": 0.8582677165354331,
+    "eval_span_recall": 0.8384615384615385,
+    "eval_span_f1": 0.8482490272373542,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 508.0,
+    "eval_class_account_number_precision": 0.9736842105263158,
+    "eval_class_account_number_recall": 0.9823008849557522,
+    "eval_class_account_number_f1": 0.9779735682819383,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.8571428571428571,
+    "eval_class_personal_handle_recall": 0.8571428571428571,
+    "eval_class_personal_handle_f1": 0.8571428571428571,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 28.0,
+    "eval_class_private_address_precision": 0.7619047619047619,
+    "eval_class_private_address_recall": 0.6666666666666666,
+    "eval_class_private_address_f1": 0.7111111111111111,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 42.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.926829268292683,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9500000000000001,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 41.0,
+    "eval_class_private_person_precision": 0.6710526315789473,
+    "eval_class_private_person_recall": 0.6257668711656442,
+    "eval_class_private_person_f1": 0.6476190476190476,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 152.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.92,
+    "eval_class_private_url_recall": 1.0,
+    "eval_class_private_url_f1": 0.9583333333333334,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 25.0,
+    "eval_runtime": 7.0151,
+    "eval_samples_per_second": 317.46,
+    "eval_steps_per_second": 2.566,
+    "epoch": 10.0,
+    "step": 3220
+  },
+  {
+    "test_loss": 0.08586616814136505,
+    "test_token_accuracy": 0.9924174456631841,
+    "test_span_precision": 0.9009708737864077,
+    "test_span_recall": 0.8560885608856088,
+    "test_span_f1": 0.8779564806054873,
+    "test_gold_spans": 542.0,
+    "test_pred_spans": 515.0,
+    "test_class_account_number_precision": 0.9752066115702479,
+    "test_class_account_number_recall": 0.9833333333333333,
+    "test_class_account_number_f1": 0.979253112033195,
+    "test_class_account_number_gold_spans": 120.0,
+    "test_class_account_number_pred_spans": 121.0,
+    "test_class_ip_address_precision": 1.0,
+    "test_class_ip_address_recall": 1.0,
+    "test_class_ip_address_f1": 1.0,
+    "test_class_ip_address_gold_spans": 9.0,
+    "test_class_ip_address_pred_spans": 9.0,
+    "test_class_personal_handle_precision": 0.9743589743589743,
+    "test_class_personal_handle_recall": 0.9743589743589743,
+    "test_class_personal_handle_f1": 0.9743589743589743,
+    "test_class_personal_handle_gold_spans": 39.0,
+    "test_class_personal_handle_pred_spans": 39.0,
+    "test_class_private_address_precision": 0.8275862068965517,
+    "test_class_private_address_recall": 0.7384615384615385,
+    "test_class_private_address_f1": 0.7804878048780489,
+    "test_class_private_address_gold_spans": 65.0,
+    "test_class_private_address_pred_spans": 58.0,
+    "test_class_private_date_precision": 0.9166666666666666,
+    "test_class_private_date_recall": 0.88,
+    "test_class_private_date_f1": 0.8979591836734694,
+    "test_class_private_date_gold_spans": 25.0,
+    "test_class_private_date_pred_spans": 24.0,
+    "test_class_private_email_precision": 1.0,
+    "test_class_private_email_recall": 1.0,
+    "test_class_private_email_f1": 1.0,
+    "test_class_private_email_gold_spans": 38.0,
+    "test_class_private_email_pred_spans": 38.0,
+    "test_class_private_person_precision": 0.7348484848484849,
+    "test_class_private_person_recall": 0.6381578947368421,
+    "test_class_private_person_f1": 0.6830985915492959,
+    "test_class_private_person_gold_spans": 152.0,
+    "test_class_private_person_pred_spans": 132.0,
+    "test_class_private_phone_precision": 1.0,
+    "test_class_private_phone_recall": 1.0,
+    "test_class_private_phone_f1": 1.0,
+    "test_class_private_phone_gold_spans": 76.0,
+    "test_class_private_phone_pred_spans": 76.0,
+    "test_class_private_url_precision": 1.0,
+    "test_class_private_url_recall": 1.0,
+    "test_class_private_url_f1": 1.0,
+    "test_class_private_url_gold_spans": 18.0,
+    "test_class_private_url_pred_spans": 18.0,
+    "test_runtime": 6.4275,
+    "test_samples_per_second": 350.37,
+    "test_steps_per_second": 2.8,
+    "epoch": 10.0,
+    "step": 3220
+  }
+]

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36d54436cb0f35aecb8cfba81f9eb72967d7a3950cff8140be9a6a9f2bbba92d
+size 2798994626

test_privacy_filter_ko.ipynb

@@ -0,0 +1,299 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "# Privacy Filter — Korean — Test Notebook\n\nTest the LoRA-fine-tuned `FrameByFrame/privacy-filter-korean` model on Korean and English PII detection.\n\n**Capabilities (9 categories):**\n- `private_person` — personal names (Korean / Western / handles)\n- `private_address` — physical / postal addresses\n- `private_phone` — phone numbers\n- `private_email` — email addresses\n- `private_date` — birthdays / personally-identifying dates\n- `private_url` — personal URLs\n- `account_number` — bank, card, RRN, passport, etc.\n- `personal_handle` — usernames / handles\n- `ip_address` — IP addresses"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Install & Load Model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Uncomment if needed\n",
+    "# !pip install transformers peft torch"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "from transformers import AutoTokenizer, AutoModelForTokenClassification\nimport torch\nimport time\n\nMODEL_ID = \"FrameByFrame/privacy-filter-korean\"\n\ntokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)\nmodel = AutoModelForTokenClassification.from_pretrained(\n    MODEL_ID, trust_remote_code=True, torch_dtype=torch.bfloat16\n)\nmodel.eval()\nif torch.cuda.is_available():\n    model.cuda()\n\nprint(f\"Model loaded. Categories: {sorted(set(v.split('-', 1)[-1] for v in model.config.id2label.values() if v != 'O'))}\")"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Helper — extract spans + show"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "\n",
+    "\n",
+    "def extract_pii(text: str, max_length: int = 512):\n",
+    "    \"\"\"Run the model on `text` and decode BIOES into character-offset spans.\"\"\"\n",
+    "    enc = tokenizer(\n",
+    "        text,\n",
+    "        truncation=True,\n",
+    "        max_length=max_length,\n",
+    "        return_offsets_mapping=True,\n",
+    "        return_tensors=\"pt\",\n",
+    "    )\n",
+    "    offsets = enc.pop(\"offset_mapping\")[0].tolist()\n",
+    "    enc = {k: v.to(model.device) for k, v in enc.items()}\n",
+    "    with torch.no_grad():\n",
+    "        logits = model(**enc).logits\n",
+    "    pred_ids = logits.argmax(-1)[0].tolist()\n",
+    "    id2label = model.config.id2label\n",
+    "\n",
+    "    spans = []\n",
+    "    active = None  # (label, start, end)\n",
+    "    for tok_idx, lid in enumerate(pred_ids):\n",
+    "        label = id2label[int(lid)]\n",
+    "        if label == \"O\":\n",
+    "            if active is not None:\n",
+    "                spans.append(active)\n",
+    "                active = None\n",
+    "            continue\n",
+    "        prefix, cat = label.split(\"-\", 1)\n",
+    "        c_start, c_end = offsets[tok_idx]\n",
+    "        if prefix == \"S\":\n",
+    "            if active is not None:\n",
+    "                spans.append(active)\n",
+    "                active = None\n",
+    "            spans.append((cat, c_start, c_end))\n",
+    "        elif prefix == \"B\":\n",
+    "            if active is not None:\n",
+    "                spans.append(active)\n",
+    "            active = (cat, c_start, c_end)\n",
+    "        elif prefix in (\"I\", \"E\"):\n",
+    "            if active and active[0] == cat:\n",
+    "                active = (active[0], active[1], c_end)\n",
+    "            else:\n",
+    "                if active is not None:\n",
+    "                    spans.append(active)\n",
+    "                    active = None\n",
+    "                if prefix == \"E\":\n",
+    "                    spans.append((cat, c_start, c_end))\n",
+    "    if active is not None:\n",
+    "        spans.append(active)\n",
+    "\n",
+    "    return [\n",
+    "        {\"label\": cat, \"start\": s, \"end\": e, \"text\": text[s:e].strip()}\n",
+    "        for cat, s, e in spans\n",
+    "        if text[s:e].strip()\n",
+    "    ]\n",
+    "\n",
+    "\n",
+    "def show(text: str):\n",
+    "    \"\"\"Detect spans and pretty-print with timing.\"\"\"\n",
+    "    t0 = time.time()\n",
+    "    spans = extract_pii(text)\n",
+    "    ms = round((time.time() - t0) * 1000)\n",
+    "    icon = \"🚫\" if spans else \"✅\"\n",
+    "    print(f\"{icon} [{ms}ms] {text[:100]}\")\n",
+    "    if spans:\n",
+    "        print(json.dumps(spans, indent=2, ensure_ascii=False))\n",
+    "    else:\n",
+    "        print(\"  (no PII detected)\")\n",
+    "    print()\n",
+    "\n",
+    "\n",
+    "def redact(text: str) -> str:\n",
+    "    \"\"\"Replace each detected span with [LABEL] in reverse order so offsets stay valid.\"\"\"\n",
+    "    spans = sorted(extract_pii(text), key=lambda s: s[\"start\"], reverse=True)\n",
+    "    out = text\n",
+    "    for s in spans:\n",
+    "        out = out[: s[\"start\"]] + f\"[{s['label'].upper()}]\" + out[s[\"end\"]:]\n",
+    "    return out"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Korean — Chat-style PII"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "show(\"김민수의 전화번호는 010-1234-5678이고 이메일은 minsu@example.com입니다.\")\n",
+    "show(\"서울특별시 강남구 테헤란로 123에 사는 박지영씨에게 연락주세요.\")\n",
+    "show(\"오늘 날씨가 좋네요.\")  # safe — no PII\n",
+    "show(\"제 생일은 1990년 5월 14일입니다. 카드번호 1234-5678-9012-3456 잊지 마세요.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Korean — Form-style document\n",
+    "\n",
+    "This is the format-matched style (`이름:`, `주소:` clues) — Privacy Filter handles it strongly because the base model was trained on similar structured PII data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "show(\"\"\"고객 정보\n",
+    "이름: 이수진\n",
+    "생년월일: 1985년 3월 12일\n",
+    "주소: 부산광역시 해운대구 우동 1457\n",
+    "연락처: 010-9876-5432\n",
+    "이메일: lee.sj@daum.net\"\"\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Korean — Banking / multi-PII"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "show(\"신한은행 계좌번호 110-234-567890 (예금주 박민수), 등록 주소 인천광역시 연수구 송도과학로 100, 비상연락 010-2345-6789, 가입일 2018.04.22.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. English — names, addresses, accounts"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "show(\"John Smith works at Google. Email: john@google.com, phone: 555-1234.\")\n",
+    "show(\"Wire to acct 110-234-567890, contact minsu@example.com\")\n",
+    "show(\"My SSN is 123-45-6789 and I live at 456 Oak Street, Springfield.\")\n",
+    "show(\"The weather is nice today.\")  # safe"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. Redaction"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "samples = [\n",
+    "    \"김민수님의 번호는 010-1234-5678입니다.\",\n",
+    "    \"서울특별시 강남구 테헤란로 123에 사는 박지영씨에게 연락주세요.\",\n",
+    "    \"My account is 110-234-567890 and email is minsu@example.com.\",\n",
+    "]\n",
+    "for s in samples:\n",
+    "    print(f\"  in: {s}\")\n",
+    "    print(f\" out: {redact(s)}\")\n",
+    "    print()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 8. Latency benchmark"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "test_cases = [\n",
+    "    (\"오늘 점심 뭐 먹지?\", 0),\n",
+    "    (\"010-1234-5678로 전화해\", 1),\n",
+    "    (\"What time is it?\", 0),\n",
+    "    (\"주민등록번호 901201-1234567\", 1),\n",
+    "    (\"김민수의 전화번호는 010-1234-5678이고 이메일은 minsu@example.com입니다.\", 3),\n",
+    "    (\"서울특별시 강남구 테헤란로 123에 사는 박지영씨에게 연락주세요.\", 4),\n",
+    "    (\"신한은행 계좌번호 110-234-567890 (예금주 박민수)\", 2),\n",
+    "    (\"My account is 110-234-567890 and email is minsu@example.com.\", 2),\n",
+    "]\n",
+    "\n",
+    "total_ms = 0\n",
+    "correct_count = 0\n",
+    "for text, expected_n in test_cases:\n",
+    "    t0 = time.time()\n",
+    "    spans = extract_pii(text)\n",
+    "    ms = round((time.time() - t0) * 1000)\n",
+    "    total_ms += ms\n",
+    "    n = len(spans)\n",
+    "    icon = \"✅\" if n == expected_n else (\"~\" if abs(n - expected_n) <= 1 else \"❌\")\n",
+    "    correct_count += int(n == expected_n)\n",
+    "    print(f\"{icon} [{ms:>4}ms] expected={expected_n} got={n} | {text[:70]}\")\n",
+    "\n",
+    "print(f\"\\nExact-count match: {correct_count}/{len(test_cases)}\")\n",
+    "print(f\"Avg latency: {total_ms/len(test_cases):.0f}ms\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 9. Custom Test\n",
+    "\n",
+    "Try your own inputs:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "show(\"여기에 한국어 텍스트를 넣으세요\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e714c627d94fd333b14f9ff32436219a4d7ac969719efe340fdc3385e1c7cd3e
+size 27868272

tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "backend": "tokenizers",
+  "eos_token": "<|endoftext|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "TokenizersBackend"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:195eed197af2b70c2fd47223db6e41715897a271a1bcbac65be2af77ec79752c
+size 4920

training_summary.json

@@ -0,0 +1,258 @@
+{
+  "checkpoint": "/models/privacy-filter",
+  "output_dir": "/workspace/data/checkpoints/ko_pii_hf_ddp_v6_lora",
+  "label_space_json": "/workspace/data/generated/ko_pii_opf_v4/label_space.json",
+  "token_labels": [
+    "O",
+    "B-private_person",
+    "I-private_person",
+    "E-private_person",
+    "S-private_person",
+    "B-personal_handle",
+    "I-personal_handle",
+    "E-personal_handle",
+    "S-personal_handle",
+    "B-private_phone",
+    "I-private_phone",
+    "E-private_phone",
+    "S-private_phone",
+    "B-private_email",
+    "I-private_email",
+    "E-private_email",
+    "S-private_email",
+    "B-private_address",
+    "I-private_address",
+    "E-private_address",
+    "S-private_address",
+    "B-private_date",
+    "I-private_date",
+    "E-private_date",
+    "S-private_date",
+    "B-private_url",
+    "I-private_url",
+    "E-private_url",
+    "S-private_url",
+    "B-account_number",
+    "I-account_number",
+    "E-account_number",
+    "S-account_number",
+    "B-ip_address",
+    "I-ip_address",
+    "E-ip_address",
+    "S-ip_address"
+  ],
+  "classifier_remap": {
+    "exact_rows_copied": 29,
+    "fallback_rows_copied": 8,
+    "random_rows_kept": 0
+  },
+  "lora": {
+    "r": 16,
+    "alpha": 32,
+    "dropout": 0.05,
+    "target_modules": [
+      "q_proj",
+      "k_proj",
+      "v_proj",
+      "o_proj"
+    ],
+    "trainable_params": 613541,
+    "total_params": 1400102970,
+    "trainable_pct": 0.0438
+  },
+  "train_dataset": {
+    "split": "train",
+    "records": 41197,
+    "tokens": 650201,
+    "spans": 11465,
+    "spans_without_token_overlap": 0,
+    "truncated_examples": 0,
+    "max_tokens": 63,
+    "records_per_path": {
+      "/workspace/data/generated/ko_pii_opf_v4/train.jsonl": 41197
+    }
+  },
+  "validation_dataset": {
+    "split": "validation",
+    "records": 2227,
+    "tokens": 34272,
+    "spans": 520,
+    "spans_without_token_overlap": 0,
+    "truncated_examples": 0,
+    "max_tokens": 45,
+    "records_per_path": {
+      "/workspace/data/generated/ko_pii_opf_v4/validation.jsonl": 2227
+    }
+  },
+  "test_dataset": {
+    "split": "test",
+    "records": 2252,
+    "tokens": 34553,
+    "spans": 542,
+    "spans_without_token_overlap": 0,
+    "truncated_examples": 0,
+    "max_tokens": 48,
+    "records_per_path": {
+      "/workspace/data/generated/ko_pii_opf_v4/test.jsonl": 2252
+    }
+  },
+  "train_metrics": {
+    "train_runtime": 2753.5271,
+    "train_samples_per_second": 149.615,
+    "train_steps_per_second": 1.169,
+    "total_flos": 1.291714856788951e+17,
+    "train_loss": 0.07032274794504509,
+    "epoch": 10.0
+  },
+  "validation_metrics": {
+    "eval_loss": 0.09874702990055084,
+    "eval_token_accuracy": 0.9907504668534081,
+    "eval_span_precision": 0.8582677165354331,
+    "eval_span_recall": 0.8384615384615385,
+    "eval_span_f1": 0.8482490272373542,
+    "eval_gold_spans": 520.0,
+    "eval_pred_spans": 508.0,
+    "eval_class_account_number_precision": 0.9736842105263158,
+    "eval_class_account_number_recall": 0.9823008849557522,
+    "eval_class_account_number_f1": 0.9779735682819383,
+    "eval_class_account_number_gold_spans": 113.0,
+    "eval_class_account_number_pred_spans": 114.0,
+    "eval_class_ip_address_precision": 1.0,
+    "eval_class_ip_address_recall": 1.0,
+    "eval_class_ip_address_f1": 1.0,
+    "eval_class_ip_address_gold_spans": 4.0,
+    "eval_class_ip_address_pred_spans": 4.0,
+    "eval_class_personal_handle_precision": 0.8571428571428571,
+    "eval_class_personal_handle_recall": 0.8571428571428571,
+    "eval_class_personal_handle_f1": 0.8571428571428571,
+    "eval_class_personal_handle_gold_spans": 28.0,
+    "eval_class_personal_handle_pred_spans": 28.0,
+    "eval_class_private_address_precision": 0.7619047619047619,
+    "eval_class_private_address_recall": 0.6666666666666666,
+    "eval_class_private_address_f1": 0.7111111111111111,
+    "eval_class_private_address_gold_spans": 48.0,
+    "eval_class_private_address_pred_spans": 42.0,
+    "eval_class_private_date_precision": 1.0,
+    "eval_class_private_date_recall": 1.0,
+    "eval_class_private_date_f1": 1.0,
+    "eval_class_private_date_gold_spans": 33.0,
+    "eval_class_private_date_pred_spans": 33.0,
+    "eval_class_private_email_precision": 0.926829268292683,
+    "eval_class_private_email_recall": 0.9743589743589743,
+    "eval_class_private_email_f1": 0.9500000000000001,
+    "eval_class_private_email_gold_spans": 39.0,
+    "eval_class_private_email_pred_spans": 41.0,
+    "eval_class_private_person_precision": 0.6710526315789473,
+    "eval_class_private_person_recall": 0.6257668711656442,
+    "eval_class_private_person_f1": 0.6476190476190476,
+    "eval_class_private_person_gold_spans": 163.0,
+    "eval_class_private_person_pred_spans": 152.0,
+    "eval_class_private_phone_precision": 1.0,
+    "eval_class_private_phone_recall": 1.0,
+    "eval_class_private_phone_f1": 1.0,
+    "eval_class_private_phone_gold_spans": 69.0,
+    "eval_class_private_phone_pred_spans": 69.0,
+    "eval_class_private_url_precision": 0.92,
+    "eval_class_private_url_recall": 1.0,
+    "eval_class_private_url_f1": 0.9583333333333334,
+    "eval_class_private_url_gold_spans": 23.0,
+    "eval_class_private_url_pred_spans": 25.0,
+    "eval_runtime": 7.0151,
+    "eval_samples_per_second": 317.46,
+    "eval_steps_per_second": 2.566,
+    "epoch": 10.0
+  },
+  "test_metrics": {
+    "test_loss": 0.08586616814136505,
+    "test_token_accuracy": 0.9924174456631841,
+    "test_span_precision": 0.9009708737864077,
+    "test_span_recall": 0.8560885608856088,
+    "test_span_f1": 0.8779564806054873,
+    "test_gold_spans": 542.0,
+    "test_pred_spans": 515.0,
+    "test_class_account_number_precision": 0.9752066115702479,
+    "test_class_account_number_recall": 0.9833333333333333,
+    "test_class_account_number_f1": 0.979253112033195,
+    "test_class_account_number_gold_spans": 120.0,
+    "test_class_account_number_pred_spans": 121.0,
+    "test_class_ip_address_precision": 1.0,
+    "test_class_ip_address_recall": 1.0,
+    "test_class_ip_address_f1": 1.0,
+    "test_class_ip_address_gold_spans": 9.0,
+    "test_class_ip_address_pred_spans": 9.0,
+    "test_class_personal_handle_precision": 0.9743589743589743,
+    "test_class_personal_handle_recall": 0.9743589743589743,
+    "test_class_personal_handle_f1": 0.9743589743589743,
+    "test_class_personal_handle_gold_spans": 39.0,
+    "test_class_personal_handle_pred_spans": 39.0,
+    "test_class_private_address_precision": 0.8275862068965517,
+    "test_class_private_address_recall": 0.7384615384615385,
+    "test_class_private_address_f1": 0.7804878048780489,
+    "test_class_private_address_gold_spans": 65.0,
+    "test_class_private_address_pred_spans": 58.0,
+    "test_class_private_date_precision": 0.9166666666666666,
+    "test_class_private_date_recall": 0.88,
+    "test_class_private_date_f1": 0.8979591836734694,
+    "test_class_private_date_gold_spans": 25.0,
+    "test_class_private_date_pred_spans": 24.0,
+    "test_class_private_email_precision": 1.0,
+    "test_class_private_email_recall": 1.0,
+    "test_class_private_email_f1": 1.0,
+    "test_class_private_email_gold_spans": 38.0,
+    "test_class_private_email_pred_spans": 38.0,
+    "test_class_private_person_precision": 0.7348484848484849,
+    "test_class_private_person_recall": 0.6381578947368421,
+    "test_class_private_person_f1": 0.6830985915492959,
+    "test_class_private_person_gold_spans": 152.0,
+    "test_class_private_person_pred_spans": 132.0,
+    "test_class_private_phone_precision": 1.0,
+    "test_class_private_phone_recall": 1.0,
+    "test_class_private_phone_f1": 1.0,
+    "test_class_private_phone_gold_spans": 76.0,
+    "test_class_private_phone_pred_spans": 76.0,
+    "test_class_private_url_precision": 1.0,
+    "test_class_private_url_recall": 1.0,
+    "test_class_private_url_f1": 1.0,
+    "test_class_private_url_gold_spans": 18.0,
+    "test_class_private_url_pred_spans": 18.0,
+    "test_runtime": 6.4275,
+    "test_samples_per_second": 350.37,
+    "test_steps_per_second": 2.8,
+    "epoch": 10.0
+  },
+  "args": {
+    "train_dataset": [
+      "/workspace/data/generated/ko_pii_opf_v4/train.jsonl"
+    ],
+    "validation_dataset": "/workspace/data/generated/ko_pii_opf_v4/validation.jsonl",
+    "test_dataset": "/workspace/data/generated/ko_pii_opf_v4/test.jsonl",
+    "label_space_json": "/workspace/data/generated/ko_pii_opf_v4/label_space.json",
+    "checkpoint": "/models/privacy-filter",
+    "output_dir": "/workspace/data/checkpoints/ko_pii_hf_ddp_v6_lora",
+    "max_length": 512,
+    "epochs": 10.0,
+    "early_stopping_patience": 3,
+    "per_device_train_batch_size": 64,
+    "per_device_eval_batch_size": 64,
+    "gradient_accumulation_steps": 1,
+    "learning_rate": 0.0005,
+    "lr_scheduler_type": "cosine",
+    "weight_decay": 0.01,
+    "warmup_ratio": 0.1,
+    "max_grad_norm": 1.0,
+    "logging_steps": 25,
+    "save_total_limit": 2,
+    "dataloader_num_workers": 4,
+    "seed": 42,
+    "overwrite_output": true,
+    "resume_from_checkpoint": null,
+    "max_train_examples": null,
+    "max_validation_examples": null,
+    "max_test_examples": null,
+    "use_lora": true,
+    "lora_r": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "lora_target_modules": "q_proj,k_proj,v_proj,o_proj"
+  }
+}