Flagship models are excellent. They are also an expensive default.
Most production traffic in the systems I ship does not need the smartest model on the market. It needs a reliable enough model with a clear promotion path when the cheap answer fails. That is when a cheaper model is the right model — not as a compromise, but as architecture.
The draft-then-promote pattern
The pattern is simple:
- Send the job to a small / cheap model (mini, Flash, Haiku-class, depending on provider and date)
- Score or validate the result (schema, rubric, retrieval citation check, confidence)
- Promote failures to a mid or flagship model
- Log both paths so you can see the mix
You pay flagship prices on the hard tail. You pay cheap prices on the fat middle. The average bill drops without pretending every ticket is a PhD oral exam.
I use this for catalog drafts, intent classification, and first-pass support answers that must still cite policy.
What “cheaper” is allowed to mean
Cheaper is not “random old model we got a coupon for.” Cheaper means:
- Good enough on a measured eval set for this task
- Stable enough JSON / tool behavior if you need structure
- Available in the regions and rate limits you need
- Documented enough that you can explain the choice in a proposal
If a bargain model fails structured outputs half the time, it is not cheaper after retries. Run the arithmetic in the AI API cost calculator with retry rate included — a 20% retry tax changes the winner.
Tasks where I start cheap on purpose
- Routing and tagging: refund vs shipping vs product question
- Extraction into a tight schema from semi-structured text
- First-draft product copy destined for human or second-model review
- Summaries of internal threads where perfect prose is optional
- Duplicate detection / clustering hints before a human merges
Tasks where I do not start on the floor:
- Legal-ish policy answers with weak retrieval
- Brand-critical homepage copy shipping without review
- Multi-step agents that can take irreversible actions
- Anything where a confident wrong answer creates chargebacks
Judgment is the product. The model card is an input.
How I measure “good enough”
Before I crown a default model, I build a tiny golden set — even 30 examples beats vibes.
Score:
- Schema validity rate
- Factual agreement with retrieved docs (for RAG)
- Rubric for tone / banned claims (for copy)
- Latency percentiles if UX is customer-facing
- Unit cost at expected volume
Only then do I pick default vs promote thresholds. “We’ll know it when we see it” is how flagship becomes the silent default forever.
Promotion triggers that work in code
Examples I wire:
- Validator failed after one repair attempt
- Retrieval similarity below threshold → do not answer cheaply; refuse or escalate
- Classifier confidence below 0.7 → mid-tier model
- Customer-facing answer that mentions prices or medical-adjacent claims → stronger model + stricter template
- User frustration signal (repeated rephrases) → handoff, not a bigger model loop
Notice handoff is sometimes better than promotion. A larger model still should not invent a tracking number.
Cost intuition with honest ranges
Suppose 50,000 support turns a month. 85% solved on a cheap model, 15% promoted.
If cheap costs roughly an order of magnitude less per turn than flagship (order-of-magnitude is the point — verify live rates), the blended bill sits much closer to cheap than to flagship. Teams that send 100% to flagship “to be safe” often pay for safety they do not measure.
Plug your own tokens and rates into the AI API cost calculator and compare:
- 100% flagship
- 100% cheap
- 85/15 blend with promotion
The blend is usually the adult conversation.
Operational gotchas
- Prompt drift: a prompt tuned for Sonnet may need shortening for a mini model
- Feature mismatch: vision, tools, or context length may force a higher tier for some jobs — route by capability, not only by price
- Cache behavior: a stable prefix helps both tiers; do not assume identical caching economics
- User-visible inconsistency: if cheap and expensive answers feel like different brands, tighten templates before you chase models
- Eval regressions: when providers ship new cheap models, re-run the golden set before you flip production
How I explain it to stakeholders
I avoid “we used the dumb model.” I say: “We use a fast specialist for routine work and a senior reviewer for exceptions.” People understand staffing analogies. They also understand invoices.
If someone insists on flagship-only, I ask which metric improves enough to justify the multiple. Sometimes brand risk makes that rational. Often it is fear without measurement. Measurement is my job in the proposal.
A catalog pipeline example
Take a nightly enrichment job for new and updated products. Most SKUs need a clean description, five bullets, and a handful of metafields. A minority have messy vendor PDFs, conflicting attributes, or brand-sensitive claims.
I draft everything on a cheap model with a strict schema. Validator passes go to a human review queue or auto-publish if the merchant opted into that risk. Validator failures, banned-claim hits, or low-confidence material tags get promoted to a stronger model once. Still failing? Dead-letter for a human — not a third model loop that burns money overnight.
That design keeps the average SKU cheap and the ugly SKU correct. It also produces logs you can show in a weekly review: promotion rate, cost per published product, top failure reasons. Those logs beat a slide that says “we use AI.”
Router design without overengineering
The router can start as boring if/else on task type. Classification jobs always cheap. Customer-facing policy answers always mid-tier with retrieval. Only later do I add a tiny classifier model to pick paths dynamically.
Overengineering the router before you have volume is a good way to spend a week inventing ML ops for a feature that runs 400 times a month. Ship the static map first. Add smart routing when the bill or the failure modes demand it.
Wherever you land, keep an escape hatch: force a model via a flag for debugging. When a merchant forwards a bad answer, you need to reproduce the path, not guess which tier woke up that day.
When cheaper becomes the wrong savings
Cut model spend last if the real waste is an oversized system prompt, missing cache, or a five-step agent for a one-step question. I have seen teams proudly move to a mini model while still pasting a 4,000-token handbook into every call with no retrieval. The invoice barely moved; quality got worse.
Sequence matters: tighten context, add caching where prefixes are stable, cap agent steps, then choose the cheapest model that still clears the eval. The AI API cost calculator helps you see which lever moves the number before you touch production defaults.
Default recommendation
Draft-then-promote: mini/Flash first, flagship only on failures — with a written eval and a cost ceiling.
That single sentence has done more for production AI budgets in my client work than any model-hot-take thread. Build the router. Keep the logs. Revisit quarterly as prices and models move.
I design these mixes from Karachi for Shopify and web teams who want AI that survives the monthly bill. If you only remember one habit from this post, remember to price the blend — not the demo model.