Chapter 1 of 5 · Fine-Tuning Language Models — When to Do It, How, and How to Know It Worked
When to fine-tune, and when a prompt or RAG is enough
Fine-tuning is not the first thing to try. The big providers themselves say start with prompting, and in 2026 fine-tuning through their APIs has shrunk sharply. This chapter helps you tell whether your problem needs a prompt, RAG or fine-tuning.
This series is for developers and small teams in Thailand thinking about “teaching” a language model their own task. Every point cites the providers’, projects’ and papers’ own documentation as checked on 25 September 2026. The examples in this series are hypothetical, to explain principles; they are not anyone’s real results.
1.1 Three ways to make a model do your task
| Approach | What it does | Fits |
|---|---|---|
| Prompting | Write better instructions and examples | Almost every problem; always try it first |
| RAG (retrieve data into the prompt) | Search your own documents and attach them for the model to read when answering | The model needs information it has never seen, such as prices or company policy |
| Fine-tuning | Train the model further on your examples | You need a consistent format or behaviour, or want a smaller model to do the same job |
The most common misunderstanding is fine-tuning so the model “knows” your data. OpenAI’s documentation says that when the model needs information from outside its training data, you put it in the prompt — the RAG approach. Data that changes often, like prices or stock, would be out of date in a fine-tuned model the moment it changed.
1.2 The providers themselves say start with prompting
- OpenAI writes that “the prompt engineering process may be all you need” to get great results.
- Google’s documentation for tuning Gemini recommends starting with prompting, then moving on to fine-tuning if required.
- OpenAI also says to invest in fine-tuning only after setting up evals (chapter 4).
The tasks OpenAI says supervised fine-tuning does well are classification, nuanced translation, generating content in a specific format, and correcting instruction-following failures. Another reason it gives is to train a smaller, cheaper, faster model to excel at one task where a large model is not cost-effective.
1.3 A hypothetical example: an online shop, two problems
Problem 1. A sports equipment shop wants a chatbot to answer questions about products, prices and its returns policy. Answer: RAG. The data changes constantly and answers must match the latest documents. Fine-tuning will not teach the model today’s prices.
Problem 2. The same shop receives many customer messages a day and wants them sorted into categories such as “product question”, “delivery tracking” and “complaint”, using a small model it can run cheaply itself. Answer: try prompting first. If the small model still misfiles messages often even with a good prompt, then fine-tune it on real messages labelled with their category. Classification with clear answers is a task fine-tuning does well.
1.4 2026: fine-tuning through APIs has shrunk
Before planning, know who still offers fine-tuning:
| Provider | What we could verify |
|---|---|
| OpenAI | Winding down its fine-tuning platform; no new users. Organisations that have not run inference on a fine-tuned model in the past 60 days can no longer create jobs, and active existing customers cannot create new jobs from 6 January 2027 |
| Google (Gemini API and AI Studio) | No model supports fine-tuning since Gemini 1.5 Flash-001 was deprecated in May 2025; still possible on Gemini Enterprise Agent Platform |
| Anthropic | No fine-tuning guide found in its current docs; only a 2024 post on fine-tuning Claude 3 Haiku in Amazon Bedrock. Claude 3 Haiku itself was retired from the Claude API on 20 April 2026, and we have not confirmed its status on Bedrock |
So for most small teams, fine-tuning in 2026 practically means fine-tuning an open-weight model with tools such as Hugging Face TRL, PEFT or Unsloth, on your own machine or a rented one. This series is written mainly from that angle.
1.5 Four questions before deciding
- Have you tried prompting seriously? Examples in the prompt, clear instructions, several variations.
- Is the problem that the model lacks information, or that it does the task in the wrong form? Missing information: RAG. Inconsistent format or behaviour: fine-tuning can help.
- Do you have enough good examples? OpenAI requires at least 10 and suggests starting with 50; Google says tuning works best with about 100 examples or more. OpenAI adds that if 50 examples have no impact, rethink the task or prompt before adding data.
- Can you measure whether it got better? If not, build an evaluation set first (chapter 4).
1.6 After this chapter
- Write your problem as one sentence, then decide whether it is “lacks information” or “wrong form”.
- Collect about 50 real examples with correct answers; they serve both to test prompts and, later, as training data.
- If you plan to use OpenAI’s fine-tuning service, check its shutdown dates first.
The next chapter is about data, the part that decides the outcome of fine-tuning more than anything else.