Most AI features do not need your flagship model on every request. They need a tiny traffic cop.
Multi-model routing is that cop: classify the job, send easy work to a cheap model, promote only the failures and the hard cases. Done well, the router pays for itself in days. Done poorly, you pay for GPT-4o to classify “is this a refund question?” a million times.
I reach for this pattern on Shopify support bots, catalog pipelines, and any agent that mixes trivia with judgment. It is the production version of “when a cheaper model is the right model” — automated.
The idea in one paragraph
Every request gets a label: simple, needs_retrieval, needs_tools, sensitive, or hard. A small model (or even rules) assigns the label. Simple generations go to mini/Flash/Haiku. Sensitive or hard generations go to a mid or flagship model. Some labels skip generation entirely and hand off to a human.
You are not building a research lab. You are building a switch statement with receipts.
Why flat “always Sonnet / always 4o” fails
Flat routing feels simpler in week one. Then volume arrives:
- Order tracking questions dominate and did not need the expensive model
- A long system prompt multiplies cost on every turn
- Retries double the bill without doubling quality
- Someone enables “agent mode” and tool loops multiply input tokens
The invoice looks like a surprise tax. Routing is how you keep the quality ceiling without paying ceiling prices for ceiling work.
Plug your mix into the AI API cost calculator — even a rough 70/20/10 split across cheap/mid/flagship changes the story versus 100% flagship.
A tiny classifier that is good enough
You do not need a fine-tuned router on day one. Options I actually ship:
- Rules first — regex/intent keywords for tracking, returns window, sizing chart links
- Cheap model JSON —
{"route":"simple|mid|flagship|handoff","intent":"..."}with 200–400 input tokens - Hybrid — rules catch the obvious 40%; model classifies the rest
Keep the classifier’s system prompt short. Cap its output tokens hard. Log route distribution weekly so you see drift.
If the classifier costs more than the savings, your classifier is too fat. That happens when people stuff the full chat history into the router. Send the latest user message plus a one-line summary instead.
Promotion, not demotion
I bias toward cheap-first with promotion:
- Draft on mini/Flash
- Run validators (schema, banned claims, retrieval citation present)
- If validators fail, retry once on a stronger model
- If that fails, hand off or refuse
Demotion (start expensive, somehow go cheaper) rarely happens in practice. Teams get shy about “risking” the cheap model and stay expensive forever. Make cheap the default path in code, not in a wiki.
Routes I use on store work
| Route | Examples | Model tier |
|---|---|---|
simple |
Intent classify, tag, short FAQ with strong retrieval | mini / Flash / Haiku |
mid |
Policy answers with citations, product Q&A | mid-tier chat |
flagship |
Messy vendor PDFs, multi-constraint bundles, angry edge threads | flagship |
handoff |
Legal, medical-adjacent, refund mutations, user asks for human | human |
Catalog copy pipelines use a cousin of this: draft cheap → checklist → promote. Support bots use it live per turn.
Guardrails the router must respect
Routing is not an excuse to freestyle on cheap models.
- Empty retrieval → refuse or handoff, regardless of model tier
- Write tools (refunds, address changes) → human confirm even if the model is flagship
- Structured outputs → validate JSON before it hits Shopify metafields
- Cost ceiling → kill switch when daily tokens exceed budget
A cheap model with refuse-when-empty is safer than a flagship model that invents shipping rules confidently.
Measuring whether routing works
Watch:
- Percent of traffic per route
- Cost per resolved conversation by route
- Promotion rate (cheap → expensive)
- User dissatisfaction / thumbs-down by route
- Escalation rate (should not spike just because you went cheap)
If promotions are rare and thumbs-down is flat, you can push more traffic cheap. If promotions and complaints cluster on one intent, fix retrieval or content before you blame the model.
For ROI conversations with merchants, pair cost savings with hours in the agent ROI calculator. Routing improves the cost side; handoff quality improves the hours-saved side.
Implementation sketch (keep it boring)
Pseudo-flow I reuse:
- Normalize inbound message
- Classify route (rules + cheap JSON)
- Retrieve if needed; abort to handoff on empty/weak
- Generate with the chosen model + schema
- Validate; promote once on failure
- Log metadata + redacted sample (see the logging playbook)
- Return answer or escalate
Put route and model name in the correlation ID logs. Future-you will thank present-you when debugging “why did this one cost $0.40?”
Common failure modes
- Router sees full history every time — costs balloon
- Too many routes — six tiers nobody understands; start with three plus handoff
- No validators — cheap model ships broken JSON into metafields
- Flagship envy — team overrides routes manually “just for launch” and never removes the override
- Ignoring 3× demo day — sale day traffic without a queue or rate limit
Sale days deserve a queue and a stricter cheap bias, not a silent upgrade to flagship for everything.
How I introduce it to merchants
I do not lead with architecture diagrams. I show two monthly bills for the same volume: all flagship vs routed. Then I show the quality checks that keep the cheap path honest. Merchants care about ceiling and risk. Routing is how you offer both.
Working from Karachi, I also care about on-call noise. Fewer expensive, flaky agent loops means fewer 1 a.m. Slack explosions when a tool retries itself into a corner.
Worked cost sketch (illustrative, not a client claim)
Suppose a support bot does 80,000 model calls a month. Roughly 70% are simple FAQs, 20% need solid mid-tier answers with retrieval, 10% are hard or sensitive.
If every call used a flagship chat model, the generation line dominates. If 70% ride a cheap tier with a short classifier in front, the same volume often lands in a completely different budget band — even after you pay for the classifier tokens. The exact dollars depend on your prompt sizes; that is why I refuse to quote a universal savings percentage. Run your tokens-per-turn through the AI API cost calculator twice: once flat, once routed.
Add retries and tool loops to both scenarios. Routing without a step cap still hurts. Routing plus caps is the combination that stays calm on sale days.
When not to bother
Skip elaborate routing when:
- Volume is tiny and flagship still costs pocket change
- Every request is already “hard” (complex document extraction only)
- You lack validators — cheap paths will ship garbage
In those cases, fix retrieval and schemas first. Return to routing when the invoice or the latency budget complains.
Bottom line
Multi-model routing is a small classifier, a promotion loop, and the discipline to keep cheap as the default. It is one of the highest-leverage cost controls available once your feature leaves the demo.
I am Faraz Hussain — Shopify, React/Next.js, and AI automation. If you want a bot or catalog pipeline with a real cost ceiling, we start with routes and validators, not with “put GPT on it.” Run your volume through the API cost calculator, then decide which percentage deserves the expensive lane.