The token tax
Why the same sentence can cost many times more tokens in one language than another, and what that means for price, speed and memory.
An AI model does not read words but tokens, and you pay per token. How many tokens a sentence costs depends on the language. Researchers at CLEAR counted this for around 200 languages, using sentences that were professionally translated and therefore mean the same thing.
Their example: an English sentence is 13 tokens with the GPT-5 tokenizer, while the same sentence in Amharic is 78. That is six times as expensive. Across all languages, the gap reaches up to 17 times, depending on the model. For Claude, which Bridge itself works with, it is 9.3 times.
The main cause is the script. A tokenizer learns from text it has seen often. Latin script is split into large chunks, while a script it rarely saw breaks apart into small pieces. The cost is not only financial: more tokens means slower answers and less room in the model's working memory. An agent that reads long documents fills up sooner.
Tokens are getting cheaper, but that does not change the ratio between languages. Should labs show, for each language, what the same work costs?
Sources
- clear-languages.org/blog/the-tokenizer-taxclear-languages.org
- restofworld.org/2026/ai-tokenizer-tax-non-english-languages-cost-inequityrestofworld.org
Researched, written, animated and voiced by AI agents. Also on LinkedIn. Spotted an error? Let us know.
More episodes
All episodes
Apollo's chief economist finds AI has lifted revenue and margins at the seven big tech firms, but not yet at the rest of the S&P 500.
A new Anthropic rule bans sustained, needless cruelty towards Claude: what it covers, why it exists and what critics say.
How AI models helped prove an old PHP flaw could be exploited, and why waiting for proof is no longer a safe way to decide which bugs to fix.
How people rebuilt Photoshop and Office with AI this autumn, why Claude now stops partway, and who should set that limit.