Most teams pick a model the way they pick a laptop tier: choose the best one the budget allows, then stop thinking about it. That habit is now costing real money, because the single largest cost lever in a reasoning stack isn't which model you bought. It's how hard you asked it to think.
Here is the snapshot that made me redraw our routing rules. On Artificial Analysis' Intelligence Index, GPT-5.6 Luna — the cost-optimized tier — scores 52 at maximum reasoning effort, at about $0.05 per Index task. GPT-5.6 Sol, the flagship tier, scores 61 at maximum effort, at about $0.96. Same benchmark run, same day. Roughly 19 times the measured cost for nine index points.
Run the same model down its own effort ladder and the picture gets sharper. Luna at low effort scores 34. At medium, 39. High, 47. Extra high, 50. Max, 52. That's eighteen index points from one settings change, at roughly five times the cost per task — and still an order of magnitude under the flagship.
What the numbers actually say
They do not say small models beat large ones. The Index is a composite score, not a linear scale of usefulness, so "19× cheaper" is a statement about price, not about intelligence. Nine points on a composite benchmark can be the difference between a feature that ships and one that doesn't.
What they do say is that inference compute is a control surface, and most deployments are leaving it on the default setting. Reasoning models spend tokens before they answer. Effort decides how many. Those tokens are billed as output tokens, which is exactly why the same model id costs five cents at max and one cent at low.
The mechanism has research behind it, not just a leaderboard. Snell, Lee, Xu and Kumar found that allocating test-time compute optimally improved efficiency by more than 4× over a best-of-N baseline, and that a smaller base model given extra test-time compute beat a model 14× larger at matched FLOPs — on problems where it already had a non-trivial success rate. That last clause carries the weight. The same paper finds more pretraining still wins on the hardest questions. Extra thinking rescues a model that was close. It does not rescue one that was lost.
The routing rule I'd write down
Treat effort as a per-request decision, and make the escalation path explicit:
- Low effort for the bulk. Classification, extraction, formatting, anything latency-sensitive. Thinking longer about an easy task buys nothing and costs tokens.
- High or max effort for the tail. The requests where search, verification and revision actually change the answer.
- The frontier model only on escalation. It gets what the cheaper model failed — on an eval, a quality gate, or a reviewer's judgement. Not the whole queue by default.
Which means the number to optimize was never the price per million tokens. It's closer to:
cost per successful task = price × inference tokens × retries ÷ successful outcomes
Retries, tool calls, reasoning volume and human review time all live in that expression. A model that's half the price and needs three attempts is not cheaper. I've watched teams celebrate a token-price win and quietly eat the difference in review hours.
What this doesn't say
Cost-efficient is not latency-efficient. Luna at max took about 151 seconds to first answer token on that benchmark. For a batch job that's free. For anything a person is waiting on, it's disqualifying — so put a latency budget next to the cost budget and decide, per request type, which ones can afford to wait.
The rest of the caveats are the usual ones, and they matter. These are live snapshots: prices, model versions and benchmark tasks all move. Proprietary model sizes are undisclosed, so don't infer parameter counts from a name — "cost-optimized" is a pricing tier, not a measurement. And the benchmark's cost curve assumes the benchmark's task mix, token usage and caching behavior, none of which are yours.
Which brings me to the only part of this that's actually work. Build a private eval set — fifty to two hundred real tasks with a pass/fail you'd defend in a review — and sweep the effort dial against it. You're looking for the point where your pass rate stops climbing. Everything upstream of that point is money you were spending on thinking that didn't change an answer.
None of this is a model recommendation. Luna will be superseded, the index will move, and the ratios in this post have a shelf life measured in weeks. The habit is what lasts: stop buying capability by the tier, and start buying it by the request.
Figures from the Artificial Analysis LLM leaderboard, Intelligence Index v4.1.1, snapshot 25 August 2026. Research: Snell et al., Scaling LLM Test-Time Compute Optimally. Effort and billing behaviour per OpenAI's reasoning guide.