Choosing a Model That Fits the Job

6.1 Start from constraints, not from a leaderboard

Answer these in order, before looking at any score.

#	Question	What the answer eliminates
1	May the data leave the organisation?	If not, every API option is gone
2	How fast must it answer?	Within a second, or is thirty seconds acceptable?
3	How many concurrent users?	Determines the runtime class and the VRAM
4	What hardware do you have?	Sets the largest possible model size
5	What is the monthly budget?	Weighs API cost against self-hosting cost
6	How much does Thai matter?	Removes models weak in Thai, however high their English scores
7	What exactly is the job?	Only now do you pick a type and size

Questions 1–6 typically eliminate more than 90% of the options before any score is consulted.

6.2 Job-to-model table

Job	Type needed	Size that suffices	Notes
Classifying text into categories	Small LLM, or a classifier	0.5–3B	This needs no large model at all. A fine-tuned small model is usually more accurate and far cheaper
Extracting data as JSON	LLM + structured output	3–8B	A runtime that enforces a schema raises accuracy sharply without a larger model
Summarising and rewriting	LLM	8–14B	For genuinely fluent Thai, size up, or pick a model trained specifically on Thai
Answering questions from documents	Embedding + Reranker + LLM	8–14B	Quality depends on retrieval more than on LLM size. Invest in retrieval first
Writing code	Coder-family LLM	14–32B and up	Size matters visibly here. Small models write code that looks right and does not run
Maths and complex logic	Reasoning model	14B and up	A mid-size reasoning model usually beats a larger ordinary one at this
Reading scans and receipts	VLM	2–8B	A small VLM trained specifically for OCR usually beats a larger general one
Transcription	ASR	Parameter count is not the metric	Choose by accuracy in your language
Semantic search	Embedding	0.3–1B	Pick one that is multilingual and whose vector dimension your database accepts
Predicting numbers from tables	Not an LLM	—	XGBoost or LightGBM: more accurate, faster, explainable, and runs on CPU

6.3 The one rule that always holds

Choose the smallest model that passes your own test set.

Not the best model on a leaderboard. A smaller model means faster answers, lower cost, room to run several at once, and VRAM left over for something else.

In practice: start with one that is obviously too small, step up until it passes, then stop. Do not start large and try to come down — you never actually come down.

6.4 Building your own eval set — the step people skip

A public leaderboard score tells you how good a model is at general work. It does not tell you how good it is at your work, and benchmark contamination keeps those scores flattering. The answer is a small test set you build yourself.

{"id":"ocr-01","input":"[receipt image]","expect":{"vendor":"Shop A","total":1250.00},"why":"normal case"}
{"id":"ocr-02","input":"[image skewed 15 degrees]","expect":{"total":890.50},"why":"handheld photo"}
{"id":"ocr-03","input":"[receipt with a watermark]","expect":{"total":2100.00},"why":"a case that failed before"}
{"id":"sum-01","input":"2000-word article...","expect_contains":["main point A","45%"],"why":"must not drop the figure"}
{"id":"neg-01","input":"question outside scope","expect_behavior":"refuse","why":"must not invent"}

How to choose the examples:

• 60% the ordinary cases you see most often
• 30% hard cases that actually failed before — the most valuable ones; capture every single one
• 10% cases the model should refuse or say it does not know

Comparing several models against one bar:

import json, time
from openai import OpenAI

MODELS = [
    ("qwen3:4b",  "http://localhost:11434/v1"),
    ("qwen3:8b",  "http://localhost:11434/v1"),
    ("qwen3:14b", "http://localhost:11434/v1"),
]

cases = [json.loads(l) for l in open("evalset.jsonl", encoding="utf-8")]

for name, base in MODELS:
    client = OpenAI(base_url=base, api_key="local")
    ok, latencies = 0, []
    for c in cases:
        t0 = time.perf_counter()
        r = client.chat.completions.create(
            model=name,
            messages=[{"role": "user", "content": c["input"]}],
            temperature=0, max_tokens=512)
        latencies.append(time.perf_counter() - t0)
        out = r.choices[0].message.content
        # The check must be automatic. Do not grade by eye
        if all(k in out for k in c.get("expect_contains", [])):
            ok += 1
    latencies.sort()
    print(f"{name:14s} passed {ok}/{len(cases)} "
          f"p50={latencies[len(latencies)//2]:.2f}s "
          f"p95={latencies[int(len(latencies)*.95)-1]:.2f}s")

Decide from that table, not from how a handful of chat turns felt.

6.5 What is specific to Thai

Issue	Effect and what to do
Inefficient tokenizer	Thai costs several times more tokens than English for the same meaning, which raises API cost, fills the context sooner, and slows generation. Test with real text to learn your chosen model’s ratio
Word segmentation	Thai has no spaces between words. A model with little Thai training segments badly and distorts meaning
High English scores do not imply good Thai	A model with excellent English can answer Thai stiffly or mix languages. Always test with real Thai text
Thai-tuned models	Both Thai-developed models and Southeast-Asia-focused ones exist, and they often beat international models of the same size on Thai. Always worth benchmarking against
Thai official and business documents	Thai numerals, Buddhist-era dates and Thai address formats usually need fine-tuning or rule support. Do not expect an international model to handle them unaided

6.6 Self-host or use an API

Aspect	Self-hosted (open-weight)	API
Data	Never leaves the machine	Goes to a provider; needs an agreement behind it
Cost	One large payment plus electricity	Pay as you go, starting near zero
Peak quality	Below the top closed models	Immediate access to the strongest models
Stability	The model does not change until you change it	The provider updates it; behaviour can shift
Customisation	Fine-tune freely	Limited to what the provider exposes
Maintenance time	High — drivers, updates, incidents	Almost none

What actually works for a small team

Start on an API to prove the idea works and to learn what the job really demands. Once you know the volume and the quality bar, move the high-volume, easy portion in-house and keep the hard, low-volume portion on the API. Most systems that work well are hybrids, not a choice of side.

6.7 Licences — check before commercial use

Licence type	What it means in practice
Apache 2.0 / MIT	Full commercial use, modification allowed. The safest for business
Developer’s own licence	Usually commercial-friendly, but with conditions: user-count ceilings, attribution requirements, or prohibited use cases. Read it properly
Non-commercial / research only	No revenue-generating use, including internal use at a company that makes money indirectly. Image models are often like this
Gated (approval required)	You must accept terms on the site before downloading, and those terms bind you

The check people forget: the licence on a modified model — fine-tuned, merged or quantized by a third party — always inherits the restrictions of its base. An uploader may write Apache 2.0, but if the base was restricted, the restriction stands. Trace it to the source before putting it near revenue.

What this chapter settles

Constraints eliminate more than 90% of the field before scores enter the conversation. Choose the smallest model that passes your own test set, not the leaderboard winner. And trace a licence back to the base model before you earn money with it.

The next chapter covers where models come from, and how to read a repository before you download from it.