Almost every company that asks us for "an AI that knows our documents" is asking for a RAG system — and almost every quote they receive is missing the same three line items. This guide gives you the real cost structure of a retrieval-augmented generation build in 2026, so you can budget properly, compare vendors on the same terms, and spot what a suspiciously cheap proposal has quietly left out.
Short answer: A working proof of concept over one clean document set runs $8K–$30K. A departmental knowledge base with citations, permissions and a real UI lands around $35K–$90K. A company-wide platform across multiple source systems is $90K–$250K. Then add $80–$15,000/month to run it, depending on scale.
1. Build Cost by RAG Tier
These are prevailing 2026 market ranges, not a specific quote. The variance inside each tier is driven overwhelmingly by two things: how many separate systems the content lives in, and how messy the documents are when they arrive.
| RAG System Tier | Build Cost | Timeline |
|---|---|---|
| Proof of concept, one clean source | $8,000 – $30,000 | 1–3 weeks |
| Departmental knowledge base with citations | $35,000 – $90,000 | 4–10 weeks |
| Customer-facing RAG assistant | $50,000 – $130,000 | 2–4 months |
| Company-wide platform, multiple sources | $90,000 – $250,000 | 3–6 months |
| Regulated / on-premise with audit trails | $250,000+ | 6+ months |
Note what does not drive the jump between tiers: the language model. Swapping one frontier model for another is a configuration change. Going from one document folder to eleven systems with different permission models is a project.
2. The Line Item Everyone Underestimates: Ingestion
Retrieval quality is capped by document quality. You cannot prompt your way out of a corpus containing four versions of the same refund policy, three of them obsolete. Before a single query is answered, someone has to parse, clean, chunk and reconcile your content — and this is where 30–50% of build effort usually goes.
| Content Type | Difficulty | Typical Prep Cost |
|---|---|---|
| Clean digital text (Markdown, Confluence, Notion) | Low | $2,000 – $8,000 |
| Structured PDFs and Office documents | Medium | $6,000 – $20,000 |
| Scanned PDFs requiring OCR | High | $15,000 – $50,000 |
| Complex tables, drawings, spreadsheets | High | $18,000 – $60,000 |
| Mixed legacy archive, unknown duplicates | Very high | $25,000 – $100,000 |
The cheapest thing you can do before requesting a quote: count your documents, note what format they are actually in, and identify who owns the authoritative version of each policy. A vendor who knows this can quote a fixed price. A vendor who does not will either pad the estimate or come back for more money in month three. Our AI readiness checklist covers exactly what to gather.
3. Monthly Running Costs
Every RAG system has two budgets: the one-time build and the monthly operating cost. Quotes that mention only the first are the single most common cause of overrun.
- Small internal system (one team, a few thousand documents): $80–$400 per month.
- Departmental system (a few hundred users, tens of thousands of chunks): $600–$2,500 per month.
- Company-wide platform (thousands of users, multiple sources): $3,000–$15,000 per month.
- Regulated / on-premise: often lower token cost but materially higher infrastructure and compliance overhead.
Those figures break down into three drivers worth understanding separately, because they scale differently:
- Vector database hosting. Scales with the number of chunks stored, not with traffic. Predictable, and often the smallest of the three.
- Embedding regeneration. Scales with how often your documents change. A static policy archive is nearly free to maintain; a live product catalogue is not.
- Generation tokens. Scales directly with queries. This is the one that grows when the system succeeds — and the one most likely to surprise you, because retrieved context makes each query far larger than the user's question alone.
Retrieval design has a direct cost consequence here. A system that stuffs twenty chunks into every prompt can cost several times more per query than one that retrieves three good ones, and it usually answers worse. Efficiency and accuracy point the same direction.
4. What a Complete Quote Should Contain
If any of these are missing, you are not comparing like with like:
- Ingestion scope — how many documents, in which formats, from which systems.
- Evaluation plan — a test set of real questions with graded answers, and a target accuracy you can hold the vendor to.
- Citation behaviour — whether every answer links back to its source, and what happens when nothing relevant is found.
- Permissions model — whether retrieval respects who is allowed to see what. Retrofitting this is expensive and it is a genuine security issue, not a nice-to-have.
- Re-indexing plan — how new and changed documents get in after launch, and who runs it.
- Both cost figures — build and monthly, with the assumptions behind the monthly number stated.
The "I don't know" test. Ask any vendor to demo what their system does with a question your documents genuinely cannot answer. A good RAG system says it does not know and offers the closest relevant source. A weak one invents a confident answer. That single demo tells you more about engineering quality than any benchmark on the proposal.
5. Estimating Payback Before You Build
The dominant ROI lever for internal RAG is search time. Staff hunting through shared drives is a cost you are already paying, just not on any invoice.
Search time saved: (staff who look things up) × (lookups per week) × (minutes per lookup) × (loaded hourly cost ÷ 60) = weekly saving.
Support deflection: (tickets/month answerable from your docs) × (cost per human-handled ticket) = monthly saving.
Error avoidance: (incidents/year caused by someone acting on an outdated document) × (cost per incident) — usually the largest number and the hardest to get signed off.
A system that turns a five-minute document hunt into a thirty-second answer across a hundred staff typically recovers its build cost within six to twelve months. If you cannot make that arithmetic work with conservative inputs, the honest answer is that you do not need a RAG system yet — and we would rather tell you that on the first call than in month four.
6. Three Ways to Spend Less Without Getting Less
- Start with one high-value document set, not everything you own. The first corpus proves the retrieval design; the second is far cheaper because the pipeline already exists.
- Fix your worst duplicates before ingestion. An afternoon of your team retiring obsolete policy versions removes days of engineering reconciliation.
- Insist on an evaluation set early. Fifty real questions with known-good answers cost almost nothing to assemble and prevent the expensive failure mode: a system that demos beautifully and is quietly wrong 20% of the time.
The Bottom Line
There is no single price for a RAG system, but there is a predictable structure: build cost scales with source systems, ingestion cost scales with document mess, and monthly cost scales with usage. Get those three estimated honestly and the budget stops being a guess.
Not sure whether you need retrieval at all, or just a good chatbot? Start with chatbot vs. RAG: which does your business actually need. If the system needs to take actions rather than answer questions, the cost structure is different again — see our AI agent development cost breakdown. And for the engineering detail behind these numbers, read building production RAG pipelines.