A model that answers instantly is a model that never checked its own work.
OpenAI put o1 at 48.9% on SWE-bench Verified, a set of real GitHub issues from open-source projects. Its successor o3, pushed to reason harder before answering, came in at 71.7%.
That gap is what reasoning models are for (o1 and o3, DeepSeek-R1, Gemini 2.5 Pro). A normal language model is a student blurting out the first thing that comes to mind: one pass, word by word, done. A reasoning model is that same student made to show their work first: trained to write out a long private chain of steps, check them, back up when one is wrong, and only then commit to the answer you see.
That thinking is not free. The private chain is made of tokens (the chunks of text a model reads and writes, and the unit you get billed in), so it costs money and it costs seconds someone spends staring at a spinner. Sorting support tickets does not need deliberation. A weird bug in code nobody remembers writing might.
Vendors call that budget test-time compute, and it is turning into a literal dial. Anthropic's newest models refuse a hand-set token budget for thinking outright: you pick an effort level from low up to max, and the model decides how long to deliberate, skipping it on easy questions at the low end.
More is not always better, either. On simple or tightly structured work, a model handed an enormous thinking budget can talk itself straight past the answer it already had.
So the skill isn't reaching for the smartest model. It's knowing which questions are worth the wait.
Quick check before you scroll: Why is it called "test-time" compute instead of "training-time" compute?
Full breakdown + the answer: frankduah.me/learnings/2026-09-12-reasoning-models-what-test-time-compute-buys-you
The answer
Because the extra computation happens when the model is actually being used (inference/"test" time) to answer a specific query, not while it's being trained on data beforehand.
New here? I post a bite-size AI / ML concept like this every day, follow me for the daily drop, and it compounds fast. Why I do it: https://lnkd.in/gK8knHDH
#ReasoningModels #AI #LLM #AIAgents #MachineLearning