Fine-Tuning

Fine-tuning takes an already trained model and continues training it on a focused dataset, adjusting its weights so it performs a specific task or follows a particular tone better than the general model does.

Why it matters

It can improve consistency on narrow jobs such as classifying support tickets or writing in a brand voice, and sometimes lets a smaller, cheaper model match a larger one. It is mainly useful for behavior and format, not as a reliable way to add fresh facts; RAG suits that better.

Example

Training examples are often prompt and ideal-response pairs:

{"prompt": "Ticket: I forgot my password",
 "completion": "category: account_access"}

A few hundred to a few thousand clean examples may be enough for a narrow task, depending on the model and goal.

Common mistakes

• Fine-tuning before trying better prompts or retrieval.
• Using noisy or inconsistent examples, which teaches the model the noise.
• Skipping a held-out test set and missing overfitting.

Related

• Training data
• Large language model
• Retrieval-augmented generation
• Inference

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.