Today's concept: fine-tuning vs prompting, when to use which

You probably don't need to fine-tune. Most people reach for it the second a model misbehaves, when a sharper prompt (or a third option most people forget) would fix it for free.

Plain-English version: prompting is talking to the model, instructions, a few examples, some context to steer it. Instant, no training, you only pay for tokens. Fine-tuning retrains the model's weights on your data so the new behavior is baked in. That costs compute, time, and a real dataset. Configuring a tool vs rebuilding the tool.

What the "fine-tune vs prompt" framing hides: there are three levers, not two.

1) Prompting, change behavior by talking. Start here, always. 2) Retrieval (RAG), the model isn't dumb, it's missing knowledge. Pull the right docs into the prompt at query time. This is how you feed it your data and today's facts. 3) Fine-tuning, you need one specific behavior, tone, or format, consistently and at scale, and prompting has plateaued.

The rule I keep in my head: RAG fixes what the model KNOWS. Fine-tuning fixes how the model ACTS. Swapping those two is the classic mistake, and fine-tuning to teach facts mostly buys you confident hallucinations.

This bites hardest in agentic loops, where every step is a prompt. One sloppy instruction that triggers retries multiplies your bill, and that's usually when people panic and fine-tune, when fixing (or caching) the prompt was the cheaper move. Fine-tuning does earn its keep when you have real signal (hundreds of clean examples, not a handful) and want to distill that behavior into a small, cheap, fast model. It's lighter than it used to be too: LoRA-style tuning nudges a tiny set of added weights, not the whole model.

One more reason to prompt first: this year tilts even further that way. Long-context windows let you drop dozens of examples straight into the prompt (many-shot). Prompt caching (KV-cache reuse) makes a big, stable system prompt cheap on repeat calls. Reasoning models think longer before answering (test-time compute), so a good prompt now clears bars that used to need training. Prompt first, measure, then spend.

Quick check before you scroll: You want your LLM to always format outputs as valid JSON. Should you fine-tune or prompt?

Full breakdown + the answer: frankduah.me/learnings/2026-06-23-fine-tuning-vs-prompting-when-to-use-which

New here? Why I post these: https://lnkd.in/gK8knHDH

#FineTuningVsPrompting #RAG #AI #LLM #AIAgents #MachineLearning