If your pipeline “usually works” until the model returns a friendly paragraph instead of data, you do not have a prompting problem. You have a contract problem.
I stopped asking models to write prose that another function must interpret with regex and hope. I ask for structured outputs — JSON that matches a schema — then validate before anything touches Shopify, a database, or a customer.
Free-text is what breaks production. Schemas are what keep it boring.
What I mean by structured outputs
Structured output means the model is constrained (by API features, tool calling, or both) to return data shaped like your application expects: fields, types, enums, nested objects. Not a chatty essay that happens to include a JSON blob somewhere in the middle.
Modern providers expose JSON schema / response format controls. Use them. If you are still doing “reply with JSON only” as a pinky-promise in the system prompt, you will eventually parse a apology.
Typical shapes I ship:
- Product enrichment: seo_title, description_html, bullets[], tags[], care
- Support routing: intent, confidence, needs_handoff, reason
- Document extraction: fields{}, missing[], warnings[]
- Moderation / QA: pass, violations[], rewrite_needed
Each of those can be validated with a schema library before the next step runs.
Why free-text breaks the pipeline
Prose fails in predictable ways:
- Markdown fences appear when you least want them
- Extra keys show up (“here’s an alternative…”)
- Types drift — a boolean becomes “yes”, a number becomes “approx 12”
- Partial answers truncate mid-JSON when max tokens is too low
- Polite refusals replace the object entirely (“I can’t help with that, but…”)
Any one of those is a page-two incident if you write straight to a metafield. Validation turns them into retries or safe fallbacks.
I would rather spend tokens on a clean retry than spend an afternoon cleaning bad catalog writes.
The pattern: schema → generate → validate → act
My default loop:
- Define the schema as the source of truth (shared with the theme/app expectations)
- Generate with provider-native structured output or tool calls
- Validate locally (required fields, enums, string lengths, HTML allowlist)
- Act only on success: write metafields, route the ticket, create the row
- On failure: retry once with the validator errors fed back, then hand off or queue
That feedback retry is important. “Your JSON failed validation: bullets must be an array of 3–5 strings” fixes more than “try again.”
Structured outputs and cost
People worry schemas cost more tokens. Sometimes the schema description adds input. In return you usually get:
- Shorter outputs (no essay wrapper)
- Fewer cascading failures
- Cheaper models that can still fill a form reliably
Filling a form is a different skill from writing a blog post. Mini / Flash / Haiku-class models often do well on schema filling when the schema is tight. Save flagship models for messy reasoning — and still ask them for JSON when the next hop is code. Model the cost in the AI API cost calculator with realistic output lengths; structured answers are often shorter than the prose they replace.
Shopify-shaped example
Suppose we enrich products for a storefront that reads metafields.
Schema (conceptual):
- seo_title: string, 20–70 chars
- bullets: array of 3–5 strings, each ≤ 120 chars
- materials: array of enums from your allowed list
- care: string
- warnings: array of strings (model can flag missing vendor data)
Validator rejects banned claims (“cures”, “guaranteed weight loss”), empty bullets, and materials outside the enum. Only then do we write custom.bullets and friends.
Editors still review brand-sensitive catalogs. Structured output does not remove editorial judgment — it removes parser archaeology.
Tool calling vs response JSON
I use both.
- Response format / JSON schema: best when the entire answer is the data object
- Tool / function calling: best when the model must choose an action (create_ticket, search_catalog, handoff) with arguments
For agents, tool arguments should be schema-validated the same way. An agent that “said” it refunded an order in prose did nothing. An agent that called refund_order with validated IDs did something — which is why permissions and confirmations matter even more once outputs are structured.
Testing that does not rely on vibes
I keep a fixture folder:
- Golden prompts with expected schema-valid outputs
- Nasty vendor inputs (empty fields, weird unicode, HTML in titles)
- Refusal cases (missing rights to answer)
Run them in CI or a nightly job. When you change the prompt or the model, you want a diff of validation pass rates — not a feeling that “it seemed better in ChatGPT.”
Also log raw model output on failures (redacted). Without that, you cannot tell whether the provider truncated, the schema was too strict, or the prompt contradicted the schema.
Common mistakes I still see
- Schema says required; prompt says “omit if unknown” — pick one and allow null or a missing[] field
- Allowing free-form HTML without a sanitizer
- Huge schemas that try to generate an entire PDP including marketing essays in one shot — split steps
- No max-length constraints, then surprised by 2,000-token “short” descriptions
- Treating validator failures as user-facing errors instead of internal control flow
Provider features change — your contract should not
JSON mode, strict schema, grammar constraints — names differ by vendor and they ship updates often. I wrap provider calls behind a small adapter so the rest of the app always sees “validated object or error.” When a provider improves structured decoding, I change one module, not twelve business workflows.
That adapter also centralizes retries, timeouts, and token accounting. If you are already modeling spend in the AI API cost calculator, wire the same per-request token estimates into logs. Structured pipelines make cost attribution easier because each job type has a predictable output shape.
Partial success and idempotency
Bulk enrichment jobs fail halfway. Plan for it.
- Process one product (or one document) per unit of work
- Store a hash of inputs + schema version so reruns skip unchanged rows
- Never mark a row complete until validation and the Shopify write both succeed
- Keep a dead-letter queue for stubborn SKUs instead of blocking the whole catalog
Idempotency matters more once outputs are structured, because it becomes tempting to “just rerun everything.” Rerunning everything without guards is how you burn budget and create conflicting metafield writes.
Human-in-the-loop without undoing the schema
Editors should not be asked to fix broken JSON. Show them a form bound to the validated object: edit bullets as a list, toggle a warning, approve or reject. On approve, write to Shopify. On reject, capture why so you can improve the prompt or the vendor input quality.
If your review UI is a giant text area of raw model output, you rebuilt free-text with extra steps. The schema should survive all the way to the human.
What I want you to ship
Stop parsing prose. Start returning JSON. Make the schema the contract between the model and your Shopify (or web) system. Validate before side effects. Retry with errors. Hand off when the contract cannot be met.
That is how AI features survive contact with a real catalog and a real support queue. It is also how you sleep when a bulk job runs overnight from Karachi while the merchant is offline.
If you are scoping an enrichment or routing feature and want the cost ceiling next to the schema, start with tokens and volume in the AI API cost calculator, then design the object you will validate on every single write.