Short answer: a RAG bot over 1,000 documents (about 800 tokens each) answering 15,000 questions a month on GPT-4o mini costs about $0.02 to embed once, ~$8.82 a month to run, and ~$106 a year. Move generation to GPT-4o and the same bot costs about $1.8k a year. The model that writes the answers is the budget. The embeddings barely register.
Start here: open the free RAG Cost for 1000 Documents Calculator, leave the defaults (1,000 docs × ~800 tokens, chunk 512, 15k queries/mo on GPT-4o mini list rates), and watch one-time embed land near $0.02, monthly query + generation near $8.82, and yearly all-in near $105.88 — then come back for why generation dwarfs embed, and how corpus size / model / re-embed cadence move the number.
What “rag cost for 1000 documents” actually means
RAG spend is not one OpenAI SKU. At a ~1000-document corpus it is usually three ledgers:
- Corpus embeddings — chunk the docs, embed once (or on a re-index cadence)
- Query-side work — embed each user question (cheap) + retrieve top-k chunks
- Generation — prompt + retrieved context in, answer tokens out (usually the real bill)
Two teams can both say “we ran RAG on a thousand docs” and pay 20× different bills if one dumps fat HTML into every chunk and answers with a frontier model. When someone asks for rag cost for 1000 documents planning, they want dollars at a stated token profile — which is why every number below locks to the calculator defaults.
The calculator defaults (reproduce in one click)
Same defaults as the live RAG cost for 1000 documents calculator:
- Documents: 1,000 · avg 800 tokens/doc
- Chunk size: 512 · overlap: 10% → chunks/doc ≈ 2
- Corpus embed tokens: 1,000 × 2 × 512 = 1,024,000
- Embed model: text-embedding-3-small at $0.02 / 1M tokens
- Re-embed cadence: never (one-shot index)
- Queries: 15,000/mo · retrieved chunks: 5 · query tokens: 80
- Generation shape: prompt 350 + 5×512 context → gen-in ≈ 2,910 · answer 250 out
- Gen rates (GPT-4o mini list approx.): $0.15 / 1M in · $0.60 / 1M out
One-time embed: 1,024,000 ÷ 1,000,000 × $0.02 ≈ $0.02
Monthly query + generation: ≈ $8.82
Yearly all-in (one embed + 12× monthly): ≈ $105.88
Disclaimer: rates above are approximate Sep 2026 list prices and change by provider, region, and volume deals. This is planning math, not an invoice or billing advice.
Why monthly query/generation dwarfs one-time embed
At 1,000 docs on 3-small, the index is basically free. The $0.02 embed is a rounding error next to Shopify apps and ads. The bill that shows up every month is generation:
- Each answered question pays for ~2,910 input tokens (prompt + five 512-token chunks) plus ~250 output tokens
- At 15k queries/mo that is tens of millions of gen tokens — not a million embed tokens
- Query embeds at 80 tokens × 15k are still cents; they do not move the P&L
So when a founder says “our RAG is expensive,” I separate three ledgers the same way the tool does: (1) corpus embed, (2) query embed, (3) generation. Nine times out of ten at ~1000 docs, (3) is the story — and the fix is tighter retrieval (fewer/shorter chunks), shorter prompts, caching, or model routing — not skipping embeddings.
Plug the same defaults into the 1000-doc RAG calculator — you should land near $0.02 embed / $8.82/mo / $105.88/yr.
Scenarios: smaller, larger, pricier gen, re-embed monthly
Scenario A — Smaller corpus (200 docs)
Hold chunking, rates, and 15k queries fixed. Scale corpus 5× down:
- Corpus tokens: 200 × 2 × 512 = 204,800 → one-time embed ≈ $0.004
- Monthly query + gen stays ≈ $8.82 (query volume did not change)
- Yearly all-in ≈ $105.79 — basically the same year as the default
Lesson: at fixed QPS, shrinking the corpus barely moves yearly spend. Obsess over generation shape, not whether you have 200 vs 1,000 docs.
Scenario B — Larger corpus (5,000 docs)
Scale corpus 5× up on the same chunk profile:
- Corpus tokens: 5,000 × 2 × 512 = 5,120,000 → one-time embed ≈ $0.10
- Monthly still ≈ $8.82 if you still retrieve only 5 chunks per query
- Yearly all-in ≈ $105.96
Embed jumps from two cents to a dime. Still not the line item. The trap at 5k docs is usually fat chunks (you silently jump from ~2 to 4+ chunks/doc) or retrieving 10–20 chunks into every prompt — that multiplies gen-in, not the embed invoice.
Scenario C — Higher generation model cost
Hold the default shape. Only swap mini list rates for a mid/frontier pair — e.g. roughly $2.50 / 1M in and $10 / 1M out (illustrative, not a live SKU quote):
- Per query gen ≈ 2,910 × $2.50/1M + 250 × $10/1M ≈ $0.0098
- At 15k queries: monthly ≈ $147 (+ tiny query embeds) vs the default $8.82
- Yearly all-in lands near $1.8k instead of ~$106
That is the real budget cliff. Mini for draft answers; escalate only when evals demand it.
Scenario D — Re-embed monthly vs never
The default is cadence never (one-shot). If your corpus churns and you full-rebuild monthly at the same 1,024,000 tokens:
- Extra embed: 12 × $0.02 ≈ $0.25/yr
- Yearly all-in moves from ~$105.88 to ~$106.13
Monthly re-embed on 3-small at 1k docs is still noise. Where re-embed hurts is when you pair full rebuilds with 3-large, multi-million-token dumps, or continuous rebuilds on every trivial edit. Rule I use: embed fields that change retrieval meaning; keep volatile ops fields out of the vector text.
What the calculator excludes
The the tool prices OpenAI-style embed + generation tokens. It does not include:
- Vector DB hosting (Pinecone, Weaviate, pgvector infra, managed Redis)
- Rerankers and cross-encoders
- Eval harnesses, labeling, and human review loops
- Prompt caching discounts or batch APIs
- App hosting, observability, or support labor
Treat ~$106/yr at the calculator defaults as the model token floor for this shape — not your fully loaded RAG product cost. If you are also embedding a Shopify catalog, pair this with the OpenAI embedding cost product catalog calculator. For raw chat API volume without retrieval, use the AI API cost calculator. The broader (non-1000-doc-focused) planner lives at /rag-cost-estimator/. The calculator here is focused on a ~1,000-document corpus.
Decision rules I use on calls
- If you only know “about a thousand docs,” compute rag cost for 1000 documents at the calculator defaults first (~$0.02 embed / ~$8.82/mo / ~$105.88/yr).
- If monthly AI spend is high but corpus embeds are still cents → audit gen model, retrieved chunk count, and prompt length — not the embedding model.
- If avg tokens/doc > ~1,500 because of raw HTML → clean text before you “need” a bigger embed budget.
- Start on a mini generation model; escalate only when your answer-quality evals justify 10×+ list-rate cost.
- Pick architecture from your doc count + chunking + QPS in the the calculator — not a LinkedIn thread.
Quick FAQ
Is ~$105.88 always the rag cost for 1000 documents yearly? Only at the calculator defaults (1k docs, 512 chunks, never re-embed, 15k queries/mo, GPT-4o mini list approx.). Change any factor and the number moves. Re-check provider pricing before you lock a board budget.
Why is embed only ~$0.02? 1,024,000 tokens on text-embedding-3-small at $0.02/1M. Embeddings at this corpus size are almost never the expensive line.
The calculator vs /rag-cost-estimator/? The calculator is focused on ~1,000 documents; the estimator covers any corpus size. Use the calculator when the question is literally rag cost for 1000 documents.
Next step
Run your real doc count, tokens/doc, chunk size, query volume, and generation rates in the RAG Cost for 1000 Documents Calculator. If you land near the calculator defaults (~$0.02 embed / ~$8.82/mo / ~$105.88/yr), your model-token floor is coherent — protect generation efficiency. If monthly is climbing while the corpus is still ~1k docs, cut retrieved chunks or route to a cheaper gen model before you blame embeddings.
Need help turning a noisy knowledge base into a scoped RAG budget a Shopify or ops team can actually afford? Work with Faraz or WhatsApp +92 334 3454747.