The "best" model is usually the wrong answer. Sending every request to the smartest, priciest model is like hiring a surgeon to put on a Band-Aid: technically capable, slow, and burning money for zero extra benefit.
Here's what I actually check.
1) Learn the ladder, not just the flagship. Every major provider ships a family, not one model. Anthropic's is Haiku 4.5, Sonnet 5, Opus 5 and Fable 5, at roughly $1/$5, $2/$10, $5/$25 and $10/$50 per million input/output tokens (Sonnet 5 is on intro pricing through Aug 31, then $3/$15). A 10x spread inside ONE family. OpenAI's GPT-5.6 has the same shape: Luna, Terra, Sol, with the cheapest tier pitched explicitly at classification, routing and high-volume work.
2) Price per token is not price per task. Tokenizers differ, so the same paragraph is a different number of tokens on different models (Anthropic's newer one emits ~30% more for identical text). And a cheap model that fumbles, retries, then escalates can easily cost more than the pricey one that got it right first time. Measure dollars per SOLVED task, not per call.
3) Tier is only one dial. Same model, less thinking: the effort setting (low up to max) trades reasoning depth for tokens, and at low effort the model can skip thinking entirely on easy inputs. That's test-time compute exposed as a plain API field. Then the unglamorous wins: prompt caching bills repeat context at 10% of input price, and the Batch API is 50% off anything that doesn't need an answer this second.
4) The router is the product. Two patterns dominate: static routing (you assign a tier per step) and cascades (cheap model first, escalate only when a check fails). The same idea recurs at every level, speculative decoding does it per token (a small model drafts, the big one verifies), distillation does it per job (train a small model on your big model's output). Match the work to the smallest thing that can do it.
5) None of it holds without evals. With no scored test set you aren't choosing a model, you're guessing and calling it architecture. Evals turn "Haiku feels fine here" into a number you can defend.
Why it matters: in agentic systems YOU write the routing logic, and that's where production cost and latency are actually won or lost.
Quick check before you scroll: Your agent handles 1,000 support messages a day, 90% are simple FAQ lookups, 10% need complex multi-step reasoning. What's the smart architecture?
The answer
Route the easy 90% to a small, cheap, fast model (like Haiku-tier), and escalate only the hard 10% to a bigger reasoning model (like Opus-tier). This "cascade" pattern keeps quality high on the cases that need it while keeping average cost and latency low.
New here? I post a bite-size AI / ML concept like this every day, follow me for the daily drop, and it compounds fast. Why I do it: https://lnkd.in/gK8knHDH