RAG vs Fine-Tuning
If the model doesn't know your facts, you want RAG. If it knows enough already but answers in the wrong style, format, or vocabulary, you want fine-tuning. RAG adds information at the moment a question is asked. Fine-tuning changes the model itself, before any question is asked. Most people arrive at this question trying to get company knowledge into a model. For that, fine-tuning is the wrong tool.
What RAG does
RAG leaves the model alone and changes its input instead. When a question comes in, the system searches your documents, pulls out the few passages that look relevant, and puts them into the request alongside the question. The model is untouched. It hasn't learned anything, and it forgets the material as soon as the request ends.
One practical consequence: nothing is locked in. Swap in a different model next month and the same retrieval setup still works. The RAG page covers the pipeline stage by stage.
What fine-tuning does
Fine-tuning takes a model that's already trained and trains it further on examples you supply, adjusting its behavior rather than what it looks up at request time. What comes out is a new version of the model that behaves differently from the one you started with. The Fine-Tuning page covers the mechanism, the real constraints (dataset size, narrowed general ability, limited model access), and what it's actually good for.
The mistake almost everyone makes: fine-tuning a model on your documents does not reliably teach it those documents. Training on text teaches patterns, not a lookup table — a model fine-tuned on a company handbook can start sounding like it while still getting details wrong, confidently, with no way to check where any answer came from. If the goal is "it should know our facts," that's a retrieval problem, not a training one.
Side by side
| RAG | Fine-tuning | |
|---|---|---|
| What changes | The input, at question time | The model itself, beforehand |
| Good at | Supplying facts the model lacks | Shaping tone, format, vocabulary |
| Bad at | Changing how the model writes | Storing facts reliably |
| Updating it | Add a document and let the system re-read it — minutes | Assemble data and retrain |
| Citations | Yes — can point at the source passage | No |
| Per-question cost | Higher — every retrieved passage is extra text the model reads, and you pay per word of it | Usually lower per call, but hosting a tuned model often costs more per word, or a fixed hourly fee |
| Access control | Filter per user at retrieval time | Not per user — you'd need a separate tuned model per audience |
| Model choice | Any model, swap freely | Limited to what your provider allows |
Which one your problem calls for
Before either one: try prompting. A clear set of instructions with a few examples solves more problems than people expect, costs nothing to change, and takes an afternoon. RAG and fine-tuning are both heavier than that. Reach for them once prompting has actually failed, not before.
Pick RAG when the answer depends on information the model was never trained on, or information that changes — company documents, product data, anything from this week. Also pick RAG when you need to show where an answer came from, or when different users are allowed to see different things. Retrieval can filter per user at query time. A fine-tuned model can't, because whatever it was trained on is inside every answer it gives, for everyone.
Pick fine-tuning when the model already has the knowledge but gets the delivery wrong: always producing one JSON shape — the structured format other software can read directly — matching a house writing style, using domain vocabulary correctly, sorting things into your categories. It's also the right tool for cost at scale. If a long prompt of instructions and examples repeats on every one of a million calls, training that behavior into a smaller model can come out both cheaper and faster.
An internal policy assistant should use RAG: policies change, staff need to see which document an answer came from, and different teams are allowed to see different policies. A service that turns messy support emails into a strict ticket format should be fine-tuned: the format never changes, thousands of past examples already exist, and the task is narrow enough that a small tuned model can beat a large general one on both cost and consistency.
Using both
They aren't alternatives, and plenty of production systems run both. Fine-tune for form, retrieve for facts. A support assistant might be fine-tuned to always answer in the company's voice and always return a structured response, while retrieval supplies the current policy text it's answering from.
Do them in that order, though — RAG first. It's cheaper, faster to change, and it often turns out to be the only part that was needed.
In this guide
FAQ
Is fine-tuning the same as training a model from scratch?
No, and the gap is enormous. Training from scratch starts from random numbers and needs a vast dataset and a budget most companies don't have. Fine-tuning needs a comparatively tiny set of examples, because the hard and expensive part was already done by whoever trained the base model.