Upload fine-tuned checkpoint: checkpoint_step1_italian_docs_v2
Browse files- README.md +127 -3
- USAGE.txt +7 -0
- config.json +124 -0
- finetune_summary.json +261 -0
- model.safetensors +3 -0
README.md
CHANGED
|
@@ -1,3 +1,127 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- it
|
| 4 |
+
license: apache-2.0
|
| 5 |
+
library_name: opf
|
| 6 |
+
base_model: openai/privacy-filter
|
| 7 |
+
pipeline_tag: token-classification
|
| 8 |
+
tags:
|
| 9 |
+
- privacy-filter
|
| 10 |
+
- pii-detection
|
| 11 |
+
- italian
|
| 12 |
+
- anonymization
|
| 13 |
+
- ner
|
| 14 |
+
- opf
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# privacy-filter-it
|
| 18 |
+
|
| 19 |
+
Fine-tuning di [openai/privacy-filter](https://huggingface.co/openai/privacy-filter) su documenti italiani sintetici per il riconoscimento di PII (Personally Identifiable Information).
|
| 20 |
+
|
| 21 |
+
Modello addestrato su dataset sintetico italiano (checkpoint_step1_italian_docs_v2).
|
| 22 |
+
|
| 23 |
+
## ⚠️ Come caricare il modello
|
| 24 |
+
|
| 25 |
+
Questo modello usa un'architettura **custom** (`model_type: privacy_filter`) **non** registrata in `transformers`. NON funziona con `AutoModel` / `transformers.pipeline`.
|
| 26 |
+
|
| 27 |
+
Per usarlo serve la libreria `opf`:
|
| 28 |
+
|
| 29 |
+
```bash
|
| 30 |
+
pip install git+https://github.com/openai/privacy-filter.git
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
```python
|
| 34 |
+
import os
|
| 35 |
+
os.environ['OPF_MOE_TRITON'] = '0' # disabilita kernel CUDA-only su MPS/CPU
|
| 36 |
+
|
| 37 |
+
from opf import OPF
|
| 38 |
+
from huggingface_hub import snapshot_download
|
| 39 |
+
|
| 40 |
+
# Scarica il modello (viene messo in cache locale)
|
| 41 |
+
local_path = snapshot_download(repo_id='capazme/privacy-filter-it')
|
| 42 |
+
|
| 43 |
+
model = OPF(
|
| 44 |
+
model=local_path,
|
| 45 |
+
device='cuda', # oppure 'mps' (Apple Silicon) o 'cpu'
|
| 46 |
+
output_mode='typed',
|
| 47 |
+
decode_mode='viterbi',
|
| 48 |
+
)
|
| 49 |
+
|
| 50 |
+
text = 'Il sottoscritto Mario Rossi, CF RSSMRA80A01H501U, residente in Via Roma 10, Milano.'
|
| 51 |
+
result = model.redact(text)
|
| 52 |
+
|
| 53 |
+
print(result.redacted_text)
|
| 54 |
+
# -> Il sottoscritto <PRIVATE_PERSON>, <CODICE_FISCALE>, residente in <PRIVATE_ADDRESS>.
|
| 55 |
+
|
| 56 |
+
for span in result.detected_spans:
|
| 57 |
+
print(f'{span.label:25s} "{span.text}" [{span.start}:{span.end}]')
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
## 📋 Categorie riconosciute
|
| 61 |
+
|
| 62 |
+
Il modello riconosce **18** categorie di PII italiane:
|
| 63 |
+
|
| 64 |
+
| Categoria | Descrizione |
|
| 65 |
+
|---|---|
|
| 66 |
+
| `private_person` | Nomi di persone fisiche |
|
| 67 |
+
| `private_address` | Indirizzi (vie, città, numeri civici) |
|
| 68 |
+
| `private_email` | Indirizzi email |
|
| 69 |
+
| `private_phone` | Numeri di telefono italiani |
|
| 70 |
+
| `private_url` | URL contenenti dati personali |
|
| 71 |
+
| `private_date` | Date (nascita, scadenze, eventi) |
|
| 72 |
+
| `account_number` | Numeri di conto (generici) |
|
| 73 |
+
| `secret` | Credenziali, password, token |
|
| 74 |
+
| `codice_fiscale` | Codice Fiscale italiano (16 caratteri) |
|
| 75 |
+
| `carta_identita` | Numero Carta d'Identità italiana |
|
| 76 |
+
| `patente` | Numero Patente di guida |
|
| 77 |
+
| `passaporto` | Numero Passaporto |
|
| 78 |
+
| `partita_iva` | Partita IVA italiana (11 cifre) |
|
| 79 |
+
| `iban` | IBAN italiano (27 caratteri) |
|
| 80 |
+
| `tessera_sanitaria` | Tessera Sanitaria |
|
| 81 |
+
| `numero_procedimento` | Numero procedimento legale (RG) |
|
| 82 |
+
| `riferimento_catastale` | Riferimento catastale (foglio/mappale) |
|
| 83 |
+
| `parte_in_causa` | Parti in procedimento giudiziario |
|
| 84 |
+
|
| 85 |
+
## 📊 Dettagli training
|
| 86 |
+
|
| 87 |
+
- **Base model**: `openai/privacy-filter`
|
| 88 |
+
- **Dataset**: sintetico, generato dal modulo `dataset_builder.py` (vedi [repo GitHub](https://github.com/capazme/privacy-filter-it))
|
| 89 |
+
- **Dati**: **7500** esempi di training, **1250** di validation, **(held-out, non usato in training)** di test (held-out)
|
| 90 |
+
- **Epoche**: 14
|
| 91 |
+
- **Batch size**: 1
|
| 92 |
+
- **Grad accum steps**: 4
|
| 93 |
+
- **Learning rate**: 1e-05
|
| 94 |
+
- **Hardware training**: cuda
|
| 95 |
+
- **Best epoch**: 14 (validation loss: 0.0000)
|
| 96 |
+
- **Param dtype**: bfloat16
|
| 97 |
+
|
| 98 |
+
## 🎯 Metriche (validation set)
|
| 99 |
+
|
| 100 |
+
- **Token accuracy** (best): 1.0000
|
| 101 |
+
- **Validation loss** (best): 0.0000
|
| 102 |
+
|
| 103 |
+
## 🎨 Esempi di output
|
| 104 |
+
|
| 105 |
+
**Input**: `Per bonifici IBAN IT60X0542811101000000123456 intestato a Luigi Bianchi. luigi.bianchi@studio.it`
|
| 106 |
+
|
| 107 |
+
**Output**: `Per bonifici <IBAN> intestato a <PRIVATE_PERSON>. <PRIVATE_EMAIL>`
|
| 108 |
+
|
| 109 |
+
## ⚖️ Licenza & limitazioni
|
| 110 |
+
|
| 111 |
+
- **Licenza**: Apache 2.0 (ereditata dal base model)
|
| 112 |
+
- **Limiti**: il dataset è sintetico — il modello potrebbe avere pattern overfitted su formati tipici (es. "CF RSSMRA80A01H501U" preceduto da prefisso). Testa con i tuoi testi prima dell'uso in produzione.
|
| 113 |
+
- **Contesto**: addestrato su testo italiano generico (email, CV, news, chat, business). Non ottimizzato per domini specifici (medico, scientifico, etc.).
|
| 114 |
+
- **Dati sintetici**: nessun dato reale di terze parti usato nel training. Tutti gli esempi sono generati programmaticamente con formati italiani validi ma valori casuali.
|
| 115 |
+
|
| 116 |
+
## 📎 Citazione
|
| 117 |
+
|
| 118 |
+
Se usi questo modello, per favore cita il lavoro originale di OpenAI:
|
| 119 |
+
|
| 120 |
+
```
|
| 121 |
+
@misc{openai-privacy-filter,
|
| 122 |
+
title = {Privacy Filter},
|
| 123 |
+
author = {OpenAI},
|
| 124 |
+
year = {2024},
|
| 125 |
+
url = {https://github.com/openai/privacy-filter}
|
| 126 |
+
}
|
| 127 |
+
```
|
USAGE.txt
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Finetuned checkpoint generated by `opf train`.
|
| 2 |
+
|
| 3 |
+
Run local inference:
|
| 4 |
+
opf --checkpoint /kaggle/working/checkpoint_step1_italian_docs_v2 --device cuda "Alice was born on 1990-01-02."
|
| 5 |
+
|
| 6 |
+
Run eval:
|
| 7 |
+
opf eval /path/to/eval.jsonl --checkpoint /kaggle/working/checkpoint_step1_italian_docs_v2 --device cuda
|
config.json
ADDED
|
@@ -0,0 +1,124 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bidirectional_context": true,
|
| 3 |
+
"bidirectional_left_context": 128,
|
| 4 |
+
"bidirectional_right_context": 128,
|
| 5 |
+
"category_version": "italian_legal_v1",
|
| 6 |
+
"default_n_ctx": 128000,
|
| 7 |
+
"encoding": "o200k_base",
|
| 8 |
+
"experts_per_token": 4,
|
| 9 |
+
"head_dim": 64,
|
| 10 |
+
"hidden_size": 640,
|
| 11 |
+
"inference_contract_version": 1,
|
| 12 |
+
"initial_context_length": 4096,
|
| 13 |
+
"intermediate_size": 640,
|
| 14 |
+
"max_position_embeddings": 131072,
|
| 15 |
+
"model_type": "privacy_filter",
|
| 16 |
+
"ner_class_names": [
|
| 17 |
+
"O",
|
| 18 |
+
"B-private_person",
|
| 19 |
+
"I-private_person",
|
| 20 |
+
"E-private_person",
|
| 21 |
+
"S-private_person",
|
| 22 |
+
"B-private_address",
|
| 23 |
+
"I-private_address",
|
| 24 |
+
"E-private_address",
|
| 25 |
+
"S-private_address",
|
| 26 |
+
"B-private_email",
|
| 27 |
+
"I-private_email",
|
| 28 |
+
"E-private_email",
|
| 29 |
+
"S-private_email",
|
| 30 |
+
"B-private_phone",
|
| 31 |
+
"I-private_phone",
|
| 32 |
+
"E-private_phone",
|
| 33 |
+
"S-private_phone",
|
| 34 |
+
"B-private_url",
|
| 35 |
+
"I-private_url",
|
| 36 |
+
"E-private_url",
|
| 37 |
+
"S-private_url",
|
| 38 |
+
"B-private_date",
|
| 39 |
+
"I-private_date",
|
| 40 |
+
"E-private_date",
|
| 41 |
+
"S-private_date",
|
| 42 |
+
"B-account_number",
|
| 43 |
+
"I-account_number",
|
| 44 |
+
"E-account_number",
|
| 45 |
+
"S-account_number",
|
| 46 |
+
"B-secret",
|
| 47 |
+
"I-secret",
|
| 48 |
+
"E-secret",
|
| 49 |
+
"S-secret",
|
| 50 |
+
"B-codice_fiscale",
|
| 51 |
+
"I-codice_fiscale",
|
| 52 |
+
"E-codice_fiscale",
|
| 53 |
+
"S-codice_fiscale",
|
| 54 |
+
"B-carta_identita",
|
| 55 |
+
"I-carta_identita",
|
| 56 |
+
"E-carta_identita",
|
| 57 |
+
"S-carta_identita",
|
| 58 |
+
"B-patente",
|
| 59 |
+
"I-patente",
|
| 60 |
+
"E-patente",
|
| 61 |
+
"S-patente",
|
| 62 |
+
"B-passaporto",
|
| 63 |
+
"I-passaporto",
|
| 64 |
+
"E-passaporto",
|
| 65 |
+
"S-passaporto",
|
| 66 |
+
"B-partita_iva",
|
| 67 |
+
"I-partita_iva",
|
| 68 |
+
"E-partita_iva",
|
| 69 |
+
"S-partita_iva",
|
| 70 |
+
"B-iban",
|
| 71 |
+
"I-iban",
|
| 72 |
+
"E-iban",
|
| 73 |
+
"S-iban",
|
| 74 |
+
"B-tessera_sanitaria",
|
| 75 |
+
"I-tessera_sanitaria",
|
| 76 |
+
"E-tessera_sanitaria",
|
| 77 |
+
"S-tessera_sanitaria",
|
| 78 |
+
"B-numero_procedimento",
|
| 79 |
+
"I-numero_procedimento",
|
| 80 |
+
"E-numero_procedimento",
|
| 81 |
+
"S-numero_procedimento",
|
| 82 |
+
"B-riferimento_catastale",
|
| 83 |
+
"I-riferimento_catastale",
|
| 84 |
+
"E-riferimento_catastale",
|
| 85 |
+
"S-riferimento_catastale",
|
| 86 |
+
"B-parte_in_causa",
|
| 87 |
+
"I-parte_in_causa",
|
| 88 |
+
"E-parte_in_causa",
|
| 89 |
+
"S-parte_in_causa"
|
| 90 |
+
],
|
| 91 |
+
"num_attention_heads": 14,
|
| 92 |
+
"num_experts": 128,
|
| 93 |
+
"num_hidden_layers": 8,
|
| 94 |
+
"num_key_value_heads": 2,
|
| 95 |
+
"num_labels": 73,
|
| 96 |
+
"param_dtype": "bfloat16",
|
| 97 |
+
"rope_ntk_alpha": 1.0,
|
| 98 |
+
"rope_ntk_beta": 32.0,
|
| 99 |
+
"rope_scaling_factor": 32.0,
|
| 100 |
+
"rope_theta": 150000,
|
| 101 |
+
"sliding_window": 257,
|
| 102 |
+
"span_class_names": [
|
| 103 |
+
"O",
|
| 104 |
+
"private_person",
|
| 105 |
+
"private_address",
|
| 106 |
+
"private_email",
|
| 107 |
+
"private_phone",
|
| 108 |
+
"private_url",
|
| 109 |
+
"private_date",
|
| 110 |
+
"account_number",
|
| 111 |
+
"secret",
|
| 112 |
+
"codice_fiscale",
|
| 113 |
+
"carta_identita",
|
| 114 |
+
"patente",
|
| 115 |
+
"passaporto",
|
| 116 |
+
"partita_iva",
|
| 117 |
+
"iban",
|
| 118 |
+
"tessera_sanitaria",
|
| 119 |
+
"numero_procedimento",
|
| 120 |
+
"riferimento_catastale",
|
| 121 |
+
"parte_in_causa"
|
| 122 |
+
],
|
| 123 |
+
"vocab_size": 200064
|
| 124 |
+
}
|
finetune_summary.json
ADDED
|
@@ -0,0 +1,261 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"artifacts": {
|
| 3 |
+
"config_json": "/kaggle/working/checkpoint_step1_italian_docs_v2/config.json",
|
| 4 |
+
"model_safetensors": "/kaggle/working/checkpoint_step1_italian_docs_v2/model.safetensors",
|
| 5 |
+
"summary_json": "/kaggle/working/checkpoint_step1_italian_docs_v2/finetune_summary.json"
|
| 6 |
+
},
|
| 7 |
+
"base_checkpoint": "/root/.opf/privacy_filter",
|
| 8 |
+
"batch_size": 1,
|
| 9 |
+
"best_epoch": 14,
|
| 10 |
+
"best_metric": 1.256955582021411e-06,
|
| 11 |
+
"best_metric_name": "validation_loss",
|
| 12 |
+
"checkpoint_category_version": "v2",
|
| 13 |
+
"device": "cuda",
|
| 14 |
+
"elapsed_s": 8991.771278478,
|
| 15 |
+
"encoding": "o200k_base",
|
| 16 |
+
"epoch_metrics": [
|
| 17 |
+
{
|
| 18 |
+
"elapsed_s": 611.6480500910002,
|
| 19 |
+
"epoch": 1,
|
| 20 |
+
"optimizer_steps": 1875,
|
| 21 |
+
"train_batches": 7500,
|
| 22 |
+
"train_loss": 0.18406920914817018,
|
| 23 |
+
"train_token_accuracy": 0.9626421135608234,
|
| 24 |
+
"train_tokens": 246106,
|
| 25 |
+
"validation_batches": 1250,
|
| 26 |
+
"validation_loss": 0.0010319853343994626,
|
| 27 |
+
"validation_token_accuracy": 0.9997995087965516,
|
| 28 |
+
"validation_tokens": 39902
|
| 29 |
+
},
|
| 30 |
+
{
|
| 31 |
+
"elapsed_s": 603.1925267070001,
|
| 32 |
+
"epoch": 2,
|
| 33 |
+
"optimizer_steps": 1875,
|
| 34 |
+
"train_batches": 7500,
|
| 35 |
+
"train_loss": 0.0006751688251858842,
|
| 36 |
+
"train_token_accuracy": 0.9999309240733668,
|
| 37 |
+
"train_tokens": 246106,
|
| 38 |
+
"validation_batches": 1250,
|
| 39 |
+
"validation_loss": 0.0004481305598764045,
|
| 40 |
+
"validation_token_accuracy": 0.9998746929978447,
|
| 41 |
+
"validation_tokens": 39902
|
| 42 |
+
},
|
| 43 |
+
{
|
| 44 |
+
"elapsed_s": 595.9333531040002,
|
| 45 |
+
"epoch": 3,
|
| 46 |
+
"optimizer_steps": 1875,
|
| 47 |
+
"train_batches": 7500,
|
| 48 |
+
"train_loss": 0.0004284777879854081,
|
| 49 |
+
"train_token_accuracy": 0.9999349873631688,
|
| 50 |
+
"train_tokens": 246106,
|
| 51 |
+
"validation_batches": 1250,
|
| 52 |
+
"validation_loss": 0.0011235129029634563,
|
| 53 |
+
"validation_token_accuracy": 0.9997995087965516,
|
| 54 |
+
"validation_tokens": 39902
|
| 55 |
+
},
|
| 56 |
+
{
|
| 57 |
+
"elapsed_s": 594.8147814399999,
|
| 58 |
+
"epoch": 4,
|
| 59 |
+
"optimizer_steps": 1875,
|
| 60 |
+
"train_batches": 7500,
|
| 61 |
+
"train_loss": 0.0004589788315978451,
|
| 62 |
+
"train_token_accuracy": 0.999910607624357,
|
| 63 |
+
"train_tokens": 246106,
|
| 64 |
+
"validation_batches": 1250,
|
| 65 |
+
"validation_loss": 0.0005426766982118603,
|
| 66 |
+
"validation_token_accuracy": 0.9997995087965516,
|
| 67 |
+
"validation_tokens": 39902
|
| 68 |
+
},
|
| 69 |
+
{
|
| 70 |
+
"elapsed_s": 594.5084605920001,
|
| 71 |
+
"epoch": 5,
|
| 72 |
+
"optimizer_steps": 1875,
|
| 73 |
+
"train_batches": 7500,
|
| 74 |
+
"train_loss": 0.00025950950850595604,
|
| 75 |
+
"train_token_accuracy": 0.9999431139427726,
|
| 76 |
+
"train_tokens": 246106,
|
| 77 |
+
"validation_batches": 1250,
|
| 78 |
+
"validation_loss": 0.0006384832375886648,
|
| 79 |
+
"validation_token_accuracy": 0.9998997543982758,
|
| 80 |
+
"validation_tokens": 39902
|
| 81 |
+
},
|
| 82 |
+
{
|
| 83 |
+
"elapsed_s": 601.9116796880007,
|
| 84 |
+
"epoch": 6,
|
| 85 |
+
"optimizer_steps": 1875,
|
| 86 |
+
"train_batches": 7500,
|
| 87 |
+
"train_loss": 0.00033364914724159483,
|
| 88 |
+
"train_token_accuracy": 0.9999390506529706,
|
| 89 |
+
"train_tokens": 246106,
|
| 90 |
+
"validation_batches": 1250,
|
| 91 |
+
"validation_loss": 0.00023689508041231628,
|
| 92 |
+
"validation_token_accuracy": 0.9999498771991379,
|
| 93 |
+
"validation_tokens": 39902
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"elapsed_s": 599.1612815619992,
|
| 97 |
+
"epoch": 7,
|
| 98 |
+
"optimizer_steps": 1875,
|
| 99 |
+
"train_batches": 7500,
|
| 100 |
+
"train_loss": 0.00021150960081863236,
|
| 101 |
+
"train_token_accuracy": 0.9999431139427726,
|
| 102 |
+
"train_tokens": 246106,
|
| 103 |
+
"validation_batches": 1250,
|
| 104 |
+
"validation_loss": 0.0001638231382475539,
|
| 105 |
+
"validation_token_accuracy": 0.9999248157987068,
|
| 106 |
+
"validation_tokens": 39902
|
| 107 |
+
},
|
| 108 |
+
{
|
| 109 |
+
"elapsed_s": 595.1891281780008,
|
| 110 |
+
"epoch": 8,
|
| 111 |
+
"optimizer_steps": 1875,
|
| 112 |
+
"train_batches": 7500,
|
| 113 |
+
"train_loss": 0.00011636061623750538,
|
| 114 |
+
"train_token_accuracy": 0.9999837468407922,
|
| 115 |
+
"train_tokens": 246106,
|
| 116 |
+
"validation_batches": 1250,
|
| 117 |
+
"validation_loss": 0.00019882116281728446,
|
| 118 |
+
"validation_token_accuracy": 0.9999248157987068,
|
| 119 |
+
"validation_tokens": 39902
|
| 120 |
+
},
|
| 121 |
+
{
|
| 122 |
+
"elapsed_s": 593.3688169760007,
|
| 123 |
+
"epoch": 9,
|
| 124 |
+
"optimizer_steps": 1875,
|
| 125 |
+
"train_batches": 7500,
|
| 126 |
+
"train_loss": 0.0001755927075592558,
|
| 127 |
+
"train_token_accuracy": 0.9999634303917824,
|
| 128 |
+
"train_tokens": 246106,
|
| 129 |
+
"validation_batches": 1250,
|
| 130 |
+
"validation_loss": 0.00022523166237503278,
|
| 131 |
+
"validation_token_accuracy": 0.9998997543982758,
|
| 132 |
+
"validation_tokens": 39902
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"elapsed_s": 593.3411462200002,
|
| 136 |
+
"epoch": 10,
|
| 137 |
+
"optimizer_steps": 1875,
|
| 138 |
+
"train_batches": 7500,
|
| 139 |
+
"train_loss": 0.0001144706639999654,
|
| 140 |
+
"train_token_accuracy": 0.9999796835509902,
|
| 141 |
+
"train_tokens": 246106,
|
| 142 |
+
"validation_batches": 1250,
|
| 143 |
+
"validation_loss": 0.00022547725828748192,
|
| 144 |
+
"validation_token_accuracy": 0.9999248157987068,
|
| 145 |
+
"validation_tokens": 39902
|
| 146 |
+
},
|
| 147 |
+
{
|
| 148 |
+
"elapsed_s": 603.6394337089987,
|
| 149 |
+
"epoch": 11,
|
| 150 |
+
"optimizer_steps": 1875,
|
| 151 |
+
"train_batches": 7500,
|
| 152 |
+
"train_loss": 0.00017106058540362378,
|
| 153 |
+
"train_token_accuracy": 0.9999634303917824,
|
| 154 |
+
"train_tokens": 246106,
|
| 155 |
+
"validation_batches": 1250,
|
| 156 |
+
"validation_loss": 5.802690870656621e-05,
|
| 157 |
+
"validation_token_accuracy": 1.0,
|
| 158 |
+
"validation_tokens": 39902
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"elapsed_s": 596.6633162110011,
|
| 162 |
+
"epoch": 12,
|
| 163 |
+
"optimizer_steps": 1875,
|
| 164 |
+
"train_batches": 7500,
|
| 165 |
+
"train_loss": 5.1230087411499746e-05,
|
| 166 |
+
"train_token_accuracy": 0.999995936710198,
|
| 167 |
+
"train_tokens": 246106,
|
| 168 |
+
"validation_batches": 1250,
|
| 169 |
+
"validation_loss": 0.0001793956412968626,
|
| 170 |
+
"validation_token_accuracy": 0.999974938599569,
|
| 171 |
+
"validation_tokens": 39902
|
| 172 |
+
},
|
| 173 |
+
{
|
| 174 |
+
"elapsed_s": 597.4834135400015,
|
| 175 |
+
"epoch": 13,
|
| 176 |
+
"optimizer_steps": 1875,
|
| 177 |
+
"train_batches": 7500,
|
| 178 |
+
"train_loss": 9.818313904666025e-05,
|
| 179 |
+
"train_token_accuracy": 0.9999756202611882,
|
| 180 |
+
"train_tokens": 246106,
|
| 181 |
+
"validation_batches": 1250,
|
| 182 |
+
"validation_loss": 0.0001664656747663711,
|
| 183 |
+
"validation_token_accuracy": 0.999974938599569,
|
| 184 |
+
"validation_tokens": 39902
|
| 185 |
+
},
|
| 186 |
+
{
|
| 187 |
+
"elapsed_s": 605.1379507809997,
|
| 188 |
+
"epoch": 14,
|
| 189 |
+
"optimizer_steps": 1875,
|
| 190 |
+
"train_batches": 7500,
|
| 191 |
+
"train_loss": 0.00013058107322193007,
|
| 192 |
+
"train_token_accuracy": 0.9999756202611882,
|
| 193 |
+
"train_tokens": 246106,
|
| 194 |
+
"validation_batches": 1250,
|
| 195 |
+
"validation_loss": 1.256955582021411e-06,
|
| 196 |
+
"validation_token_accuracy": 1.0,
|
| 197 |
+
"validation_tokens": 39902
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"elapsed_s": 597.8637465249994,
|
| 201 |
+
"epoch": 15,
|
| 202 |
+
"optimizer_steps": 1875,
|
| 203 |
+
"train_batches": 7500,
|
| 204 |
+
"train_loss": 3.845093296363575e-06,
|
| 205 |
+
"train_token_accuracy": 1.0,
|
| 206 |
+
"train_tokens": 246106,
|
| 207 |
+
"validation_batches": 1250,
|
| 208 |
+
"validation_loss": 1.754828393192001e-05,
|
| 209 |
+
"validation_token_accuracy": 1.0,
|
| 210 |
+
"validation_tokens": 39902
|
| 211 |
+
}
|
| 212 |
+
],
|
| 213 |
+
"epochs": 15,
|
| 214 |
+
"generated_at_unix": 1777048760.4942422,
|
| 215 |
+
"grad_accum_steps": 4,
|
| 216 |
+
"label_space_json_path": "/kaggle/working/custom_label_space.json",
|
| 217 |
+
"label_space_source": "label-space-json",
|
| 218 |
+
"learning_rate": 1e-05,
|
| 219 |
+
"max_grad_norm": 1.0,
|
| 220 |
+
"num_output_labels": 73,
|
| 221 |
+
"num_train_examples": 7500,
|
| 222 |
+
"num_train_windows": 7500,
|
| 223 |
+
"num_validation_examples": 1250,
|
| 224 |
+
"num_validation_windows": 1250,
|
| 225 |
+
"output_checkpoint_dir": "/kaggle/working/checkpoint_step1_italian_docs_v2",
|
| 226 |
+
"output_head_reinitialized": true,
|
| 227 |
+
"output_head_rows_copied": 73,
|
| 228 |
+
"output_head_rows_copied_exact": 33,
|
| 229 |
+
"output_head_rows_copied_fallback": 40,
|
| 230 |
+
"resolved_category_version": "italian_legal_v1",
|
| 231 |
+
"resolved_n_ctx": 128000,
|
| 232 |
+
"schema_version": 1,
|
| 233 |
+
"serialized_param_dtype": "bfloat16",
|
| 234 |
+
"span_class_names": [
|
| 235 |
+
"O",
|
| 236 |
+
"private_person",
|
| 237 |
+
"private_address",
|
| 238 |
+
"private_email",
|
| 239 |
+
"private_phone",
|
| 240 |
+
"private_url",
|
| 241 |
+
"private_date",
|
| 242 |
+
"account_number",
|
| 243 |
+
"secret",
|
| 244 |
+
"codice_fiscale",
|
| 245 |
+
"carta_identita",
|
| 246 |
+
"patente",
|
| 247 |
+
"passaporto",
|
| 248 |
+
"partita_iva",
|
| 249 |
+
"iban",
|
| 250 |
+
"tessera_sanitaria",
|
| 251 |
+
"numero_procedimento",
|
| 252 |
+
"riferimento_catastale",
|
| 253 |
+
"parte_in_causa"
|
| 254 |
+
],
|
| 255 |
+
"train_dataset": "/kaggle/working/datasets/step1_train.jsonl",
|
| 256 |
+
"train_dataset_variant": "full",
|
| 257 |
+
"validation_dataset": "/kaggle/working/datasets/step1_val.jsonl",
|
| 258 |
+
"validation_dataset_variant": "full",
|
| 259 |
+
"validation_split": null,
|
| 260 |
+
"weight_decay": 0.01
|
| 261 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:420e24586f4776bba6fefd91d57e1875ddcd3c6040f674ded6c172a1048b1f0a
|
| 3 |
+
size 2799035176
|