Corianas
/

char128_shift_tokenizer

Model card Files Files and versions

Corianas commited on Oct 15, 2025

Commit

c1010b0

·

verified ·

1 Parent(s): 0e30278

Update tokenizer.json

Files changed (1) hide show

tokenizer.json +1 -1

tokenizer.json CHANGED Viewed

@@ -342,7 +342,7 @@
       "^": 68,
       "_": 69,
       "?": 70,
-      "è": 71,
       "é": 72,
       "¤69": 73,
       "¤70": 74,

       "^": 68,
       "_": 69,
       "?": 70,
+      "%": 71,
       "é": 72,
       "¤69": 73,
       "¤70": 74,