rtferraz
/

domainTokenizer

Model card Files Files and versions

domainTokenizer / src /domain_tokenizer

Ctrl+K

Ctrl+K

1 contributor

History: 6 commits

rtferraz's picture

Add domain_tokenizer.py — DomainTokenizerBuilder (core assembler, HF integration)

818a2e9 verified 10 days ago

schemas
Add schemas package init 10 days ago
tokenizers
Add domain_tokenizer.py — DomainTokenizerBuilder (core assembler, HF integration) 10 days ago
__init__.py

895 Bytes
Phase 2A: Core tokenizer library — schema, field tokenizers, composite builder, predefined schemas, 72 passing tests 10 days ago
schema.py

7.49 kB
Add schema.py — DomainSchema, FieldSpec, FieldType definitions 10 days ago