Chapter 3 of 5 · Fine-Tuning Language Models — When to Do It, How, and How to Know It Worked

LoRA and QLoRA in practice: choosing a Thai base model and tools

LoRA trains only small matrices inserted into the model; QLoRA does the same on a model compressed to 4 bits, making fine-tuning possible on a single GPU. This chapter covers the idea, the trade-off, the Thai base models available with their licences, and the current tools.

With data in hand, the next question is how to train. Training every parameter of a model (full fine-tuning) needs a lot of memory, so most small teams use LoRA or QLoRA.

3.1 LoRA: leave the original weights alone, train a small add-on

The LoRA paper (2021) describes the method as freezing the model’s pre-trained weights and injecting small trainable matrices into each layer. Against full fine-tuning of GPT-3 175B with Adam, it reports:

10,000 times fewer trainable parameters;
3 times less GPU memory;
no added latency once the add-on is merged into the model (chapter 5).

What training produces is an adapter: a small file that always has to be used with the same base model.

3.2 QLoRA: LoRA on a model compressed to 4 bits

The QLoRA paper (2023) compresses the base model to 4 bits with a new data type, NF4, and trains LoRA on top, with two more memory savers: double quantization and paged optimizers. It reports fine-tuning a 65B model on a single 48GB GPU while matching 16-bit fine-tuning performance.

3.3 The trade-off: LoRA learns less, but forgets less

Biderman et al. (2024) compared LoRA with full fine-tuning in two domains, programming and maths, using standard low-rank settings, and concluded that LoRA substantially underperforms full fine-tuning on the target task, but better maintains the base model’s performance on tasks outside the target domain.

Your task may behave differently. For a small team teaching one specific task without losing general ability, that trade is often worth it — but always measure (chapter 4).

3.4 Choosing a base model: Thai support and licences

Options with official model cards, from their Hugging Face cards on the date checked:

Model	Built on	Licence per card	Notes
Typhoon 2.5 (typhoon-ai)	Qwen3 4B and 30B-A3B	Apache 2.0	The card links a Qwen licence file; read both
Typhoon 2.1	Gemma3 12B and 4B	Gemma licence	Gemma’s restrictions pass on to derived models (chapter 5)
ThaiLLM-8B	Qwen3-8B-Base	Apache 2.0	A base model, not yet instruction-tuned; the card recommends LLaMA-Factory for that step
Pathumma (NECTEC) 4B	ThaiLLM	Apache 2.0	Post-training data is not public
SEA-LION v4.8 (AI Singapore)	NVIDIA Nemotron 3	MIT	Thai is one of 8 languages; NVIDIA’s underlying model has its own licence — read it too
OpenThaiGPT 1.6 72B	Qwen2.5-72B-Instruct	Qwen licence	Large; needs a lot of resources

Licences one layer down matter too:

Not every recent Qwen is Apache 2.0. Qwen3.8-27B is; Qwen3.8-Flash-Next uses the Qwen Community License, under which a Model-as-a-Service or AI work assistant business needs a separate licence.
Gemma 4 is Apache 2.0; earlier Gemma models are under the Gemma Terms of Use.
Llama 4 requires “Built with Llama” when distributing; a fine-tuned Llama you distribute must have a name beginning with “Llama”; and a separate licence is needed above 700 million monthly users as of the Llama 4 release date.

On tokenizers: the first Typhoon paper (2023) reported that adding 5,000 Thai tokens made its tokenizer 2.62 times more efficient on Thai than GPT-4’s; Typhoon 2 no longer extended the tokenizer. That does not say which approach is right, but it shows that Thai token counts vary a lot between models, which affects how much text fits and what it costs.

3.5 Tools (latest releases as of 25 September 2026)

Tool	What it does	Version
Transformers	Loads and runs models	5.17.0
PEFT	LoRA and other parameter-efficient methods	0.21.0
TRL	Training library, including SFTTrainer for supervised fine-tuning	1.13.0
bitsandbytes	4- and 8-bit quantization (behind QLoRA)	0.50.2
Unsloth	Faster training with less memory	2026.9.11
Axolotl	Training from a config file	0.19.0
LLaMA-Factory	Training many models from one place	0.9.5

These tools release often, and parameter names change between versions. Follow the examples in the documentation for the version you actually installed; don’t copy code from older articles, including this one.

3.6 The overall steps

Choose a base model whose licence fits your use and whose Thai is good enough for your task.
Prepare data as messages, split into training and test sets (chapter 4).
Load the base model; if memory is short, load it in 4 bits for QLoRA.
Configure LoRA with PEFT and train with TRL’s SFTTrainer, which applies the chat template for you.
Evaluate on the test set against the base model with a prompt only.
Merge the adapter into the model and deploy it (chapter 5); after QLoRA, reload the base model in 16 bits before merging.

3.7 A hypothetical example: pick for the job

The shop from earlier chapters wants a model to sort customer messages, running on one machine in the shop. The messages mix Thai and English, and the use is commercial.

A sensible choice is a small model whose licence allows commercial use and which already has Thai instruction tuning — say Typhoon 2.5 4B from the table in 3.4 — trained with QLoRA. Choosing a 72B model because “bigger must be better” would make running costs far too high for sorting into a handful of categories.

3.8 After this chapter

Write down three things about your chosen base model: what it is built on, its licence, and the licence of the model underneath.
Install the current tool versions, then open the examples in that version’s documentation.
Run training on a small slice of data first to be sure the whole pipeline works, then use the full set.

The next chapter covers measuring whether the trained model really improved, and did not lose anything.