README.md

metadata

title: Token Counter
emoji: 🔢
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.49.1
python_version: '3.12'
app_file: app.py
pinned: false
license: mit
short_description: Count tokens across Claude, GPT, Llama tokenizers.
tags:
  - tokenization
  - llm
  - context-window
  - agentfit

Token Counter

Paste any text and see how it tokenizes across Claude, GPT, and other model families. Powered by agentfit.

Why?

• Different tokenizers split the same string very differently — Claude often uses ~half the tokens GPT does for the same Chinese / emoji input.
• Useful when budgeting prompts and deciding which model to use for non-English content.
• Sanity-check your own token counter against a reference.

Related

• agentfit on PyPI
• The Agent Reliability Stack
• Companion dataset: token-counting-edge-cases