Most of what an AI agent sends to the biggest, most expensive model available is clerical work. Pull the invoice number off this page. Pick one of five labels. Return the answer in exactly the right format.
Genius rates, for filing.
An agent here is just a program that calls a language model over and over to finish a multi-step job. NVIDIA researchers counted those calls (Belcak et al., 2025): most never needed a giant model.
A small language model means roughly 1 to 14 billion parameters, against the hundreds of billions in a frontier model like GPT-5 or Claude. Parameters are the tunable numbers inside a model that hold what it has learned: more of them buys broader knowledge, and costs more money and more waiting per response.
Here is the counterintuitive part. Broad knowledge is not what a field-extraction step needs. It needs the same correct shape every time. Fine-tune a 7-billion-parameter model on that one job (show it thousands of examples until the shape is second nature) and it will hit that shape more reliably than a generalist that can do far more and occasionally gets inventive.
Across three open-source agent systems they estimated a specialised small model could take over 40% to 70% of the calls. And serving a 7-billion-parameter model runs 10 to 30 times cheaper than a 70 to 175 billion one: less waiting, less energy, less hardware.
The trap is reading that as "use small models". Swap them in everywhere and the genuinely hard steps degrade quietly: no error, just worse decisions. The real work is measuring which calls are narrow and which need actual reasoning, then routing each one to the right size.
Big model as the planner. Small models as the crew that never has an off day.
Quick check before you scroll: Why might a 3B-parameter model outperform a 500B-parameter model on a specific agent task like extracting fields from an invoice?
Full breakdown + the answer: frankduah.me/learnings/2026-09-10-small-language-models-when-smaller-wins
New here? I post a bite-size AI / ML concept like this every day, follow me for the daily drop, and it compounds fast. Why I do it: https://lnkd.in/gK8knHDH
#SmallLanguage #AI #LLM #AIAgents #MachineLearning
The answer
Because that task is narrow and repetitive, the small model can be fine-tuned tightly on exactly that job, making it sharper "in its lane" even though the giant model knows vastly more in general.