KoishiAI Lab · measured 27 October 2026
How many more tokens does Thai take than English?
Most AI APIs charge by the token, not by the word. We measured how many more tokens the same text takes in Thai than in English, with three real tokenizers, on 1,997 news sentences translated into Thai one by one.
Results
| Tokenizer | Thai ÷ English (median) | Typical range (10–90%) | Tokens per character (Thai text) |
|---|---|---|---|
| GPT (OpenAI’s o200k tokenizer) | 1.83× | 1.35–2.41× | 0.400 |
| Typhoon 2.5 and Qwen3 (the same tokenizer) | 2.39× | 1.74–3.15× | 0.535 |
| Qwen3.6 | 1.16× | 0.89–1.52× | 0.261 |
The median is taken over per-sentence ratios. The typical range is the middle 80% of sentences, leaving out the 10% with the lowest and the 10% with the highest ratio.
What it means in practice
- If a prompt takes 1,000 tokens in English, the same prompt in Thai takes about 1,800 tokens on GPT and about 2,400 on Typhoon 2.5. On an API billed per token, that part of the bill rises in the same proportion, and it uses that much more of the context window.
- Per-token prices differ between models, so these ratios cannot compare prices across models on their own; multiply by each model’s price first.
- Qwen’s newer tokenizer narrows the gap to 1.16×. Typhoon 2.5 is a Thai model but uses Qwen3’s tokenizer unchanged, so Thai takes the most tokens of the three.
See each model’s price per Thai character in the Thai AI price tool →
What our own articles show
- We repeated the measurement on the 173 article pairs on this site published in both Thai and English: medians of 1.99× on GPT, 2.53× on Typhoon 2.5 and 1.23× on Qwen3.6 — about 6–9% above the translated sentences.
- We do not use our articles as the main figure: the Thai and English versions are written separately, not translated sentence by sentence, so the ratio depends on how each version was written and changes over time. On GPT the monthly median ranged from 1.31 to 2.10 (some months have fewer than 10 pairs).
- Our articles use about 4–5% fewer tokens per character than the news sentences, probably because they mix in more English.
The figure we used before
Before 27 October 2026 the tools on this site used a figure measured on 23 September 2026 from 109 article pairs: 1.67× on GPT, 1.99× on Typhoon 2.5 and 1.20× on Qwen3.6. We did not keep the script for that measurement, so it cannot be repeated as it was; measuring the 133 pairs published by then with the documented method gives 1.86 on GPT, not 1.67. The old figures were about 9% lower than the translated sentences on GPT and about 17% lower on Typhoon 2.5, but about 4% higher on Qwen3.6. Every tool now uses the new figures.
Method
- Main data: Microsoft’s NTREX-128 set — 1,997 English news sentences with human Thai translations, files newstest2019-src.eng.txt and newstest2019-ref.tha.txt.
- Tokens counted sentence by sentence with each family’s real tokenizer, no special tokens; the ratio is Thai tokens over English tokens per sentence.
- Tokens per character = total tokens ÷ total characters in each language.
- Our articles: only articles published in both languages; Sources section removed, links reduced to their text, images left out.
Limits
- The translated sentences are general news from 2019. Specialised text or conversation may give different figures.
- Claude, Gemini and other families were not measured here. Our token counter estimates them from a separate measurement through their APIs.
- Tokenizers change with new model releases. We will measure again when a widely used tokenizer changes.
Raw data
- Summary from the translated sentences (JSON) — Derived from NTREX-128, licensed CC BY-SA 4.0
- Per-article results, 173 pairs (CSV) — Licensed CC BY 4.0
- Per-article results with summary (JSON) — Licensed CC BY 4.0
Translated sentences: NTREX-128 by Microsoft (CC BY-SA 4.0) — github.com/MicrosoftTranslator/NTREX