Skip to content
KoishiAI
ไทย

Thai Text Cleaner for AI

Thai copied out of PDFs often looks fine but is encoded wrongly underneath, so AI misreads it and spends more tokens. Paste it here and copy the fixed version.

Your text never leaves your machine — fixed in the browser
Try:
Tokens (GPT) · before
–
after
–
saved
–

What was fixed

    • 0
    • 0
    • 0
    • 0
    • 0
    • 0
    • 0

    More options (off by default — they change what you see)

    • 0
    • 0
    • 0
    • 0

    Why clean before sending to AI

    Text that looks identical but is encoded differently reads as different words to a model, and splits into more tokens. Counted with GPT’s o200k tokenizer:

    CorrecttokensMis-encodedtokensWhat differs
    ทำงาน 2 ทํางาน 4 ํ + า instead of ำ
    น้ำ 1 น้ํา 3 ํ + า instead of ำ
    ที่นี่ 2 ท่ีน่ี 5 ่ typed before ี
    แปลก 3 เเปลก 4 เ + เ instead of แ
    หน้า 1 หนา 5 U+F70B (PUA) instead of ้ — renders as nothing

    First 700 characters of a Royal Gazette PDF: 453 tokens before, 324 after.

    Worth knowing

    • Many Thai PDFs use old fonts that keep repositioned tone marks in the Private Use Area (U+F70A is mai ek shifted down). Copying yields characters a model does not know. This tool maps them back using the fonts’ own tables, Windows and Mac.
    • NFKC normalization, which many programs apply, splits sara am into nikhahit + sara aa by itself. Use NFC for Thai.
    • Some PDFs put a space before a vowel or tone mark (“สมเด ็จ”). This tool drops the space and rejoins “สมเด็จ”. PyThaiNLP drops the mark instead, giving “สมเด จ”.
    • “(cid:123)” or � means the letters were lost at copy time; only OCR from the image can recover them.

    How it was tested

    • The defaults only fix what a reader cannot see: encodings, invisible characters, typing order. Joining lines, collapsing spaces and converting Thai digits are opt-in.
    • Run over all 187 Thai article files on this site, drafts included (2.2 million characters, 2026-09-24), the defaults changed exactly one spot — a real typo (a doubled mai ek).
    • On text copied from a Royal Gazette PDF, the output matches a version checked line by line by eye.
    • Joiners used by emoji (ZWJ) are removed only next to Thai letters, so family and flag emoji survive.
    • Most rules are adapted from PyThaiNLP (Apache-2.0); the PUA table comes from the OpenType shaping documents and linux.thai.net.

    FAQ

    Is my text sent anywhere?

    No. Everything runs in your browser, the token counter included.

    Why does copying from a PDF break Thai?

    A PDF stores glyphs, not letters. Old Thai fonts have several positioned glyphs per tone mark, each with its own code, so copying yields the glyph’s code instead of the real tone mark.

    Does cleaning change the meaning?

    The defaults change nothing visible — only the underlying encoding. Options like joining lines or converting Thai digits change the text’s form, so they start off.

    Can I use it on text with code?

    Yes. The defaults leave spaces and line breaks alone. Keep “repeated spaces” off for code — it would wreck indentation.