Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Log In
Sign Up
Ezekiel999
/
aksara-tokenizer-20b
like
0
5 languages
tokenizer
bpe
aksarallm
indonesian
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
main
aksara-tokenizer-20b
9.54 MB
Ctrl+K
Ctrl+K
1 contributor
History:
3 commits
Ezekiel999
Upload 131K BPE tokenizer (vocab=131072, fertility id=1.22 en=1.28)
01be583
verified
13 days ago
.gitattributes
Safe
1.52 kB
initial commit
13 days ago
README.md
Safe
1.83 kB
Upload 131K BPE tokenizer (vocab=131072, fertility id=1.22 en=1.28)
13 days ago
special_tokens_map.json
Safe
106 Bytes
Upload 131K BPE tokenizer (vocab=131072, fertility id=1.22 en=1.28)
13 days ago
tokenizer.json
Safe
9.54 MB
Upload 131K BPE tokenizer (vocab=131072, fertility id=1.22 en=1.28)
13 days ago
tokenizer_config.json
Safe
534 Bytes
Upload 131K BPE tokenizer (vocab=131072, fertility id=1.22 en=1.28)
13 days ago
