When to fine-tune — and how to know it worked

Fine-tuning is the expensive last resort, not the first lever. Before you spend the GPU budget, work through this checklist. We have stopped a lot of unnecessary fine-tunes by walking clients through it.

Try these first, in order

1. Better prompt. Better instructions, better examples, better structure. Often closes 50% of the quality gap.
2. Better retrieval. If the model lacks knowledge, give it knowledge. Cheaper and more controllable than baking it into weights.
3. Better eval. You cannot improve what you cannot measure. Build the eval; iterate against it.
4. A bigger model. Sometimes the answer is just to call a stronger model. Re-test cost vs. quality.
5. Decompose the task. Two model calls with cleaner prompts often beat one fine-tuned call.
6. Tool use. If the model needs to do math, give it a calculator. Don't fine-tune for arithmetic.

When to fine-tune anyway

• Style or voice that prompting can't pin down (legal language, brand tone, specific jargon).
• Latency-sensitive workloads where a small fine-tuned model can replace a larger generic one.
• Cost-sensitive workloads at high volume where the savings amortize the training cost in months.
• Behavior the model resists. Sometimes the alignment training of the base model fights you. Fine-tuning realigns it.

How to know it worked

Run your eval suite before and after. Quality must improve on the eval set, hold or improve on a held-out set, and not regress on safety/refusal evals. If any of those three fail, the fine-tune is not ready for production.

What teams underestimate

1. Data prep. 80% of the work. Cleaning, formatting, deduplication, labeling — the model is the easy part.
2. Catastrophic forgetting. Fine-tuning on your task can degrade the model's other capabilities. Mix in general-purpose data.
3. Maintenance. Every base-model upgrade requires re-fine-tuning. Budget for this on an ongoing basis.