Retrieval architecture for production AI not a feature you switch on
Why retrieval belongs in the architecture rather than bolted onto a chat box, and what it takes to run it in production.
What this covers.
Most AI pilots stall because retrieval was treated as an integration detail. This paper sets out the architecture we use on live systems: how documents are chunked and versioned, how retrieval is evaluated before a model is chosen, where guardrails sit, and how the whole thing is observed once real users are on it.
Evaluation before model choice
A scored retrieval set built from real questions, so model swaps become a measurable decision rather than a preference.
Guardrails at the boundary
Permission-aware retrieval, refusal behaviour and citation rules defined as system behaviour, not prompt text.
Observability from day one
Query, retrieval and answer logging that lets you explain any response weeks after it was given.
Section by section.
Where pilots break
The four failure patterns we see most: unversioned corpora, unscoped permissions, no evaluation set, and no owner after launch.
Reference architecture
Ingestion, chunking, embedding, retrieval, ranking, generation and audit — with the decisions that matter at each layer.
Evaluation harness
How to build a scored question set, what to measure, and the thresholds we hold before anything reaches users.
Operating model
Who owns the corpus, how often it is refreshed, and what an on-call rotation for an AI system actually covers.
Three things you can act on.
- A reference architecture you can hold your vendor or team to
- An evaluation approach that makes model selection defensible
- The operating costs and roles a production AI system needs
The detail before you ask for it.
Format
Whitepaper · PDF · 18 pages
Category
AI engineering
Extent
18 pages
Written for
CTOs, heads of engineering and data leaders
Level
Practitioner
Language
English
Published
2026-02-11
Author
Lucidspire IT Services practice
Practices behind this resource.
Where it applies most.
Work behind the thinking.
Take it with you.
Retrieval architecture for production AI
Why retrieval belongs in the architecture rather than bolted onto a chat box, and what it takes to run it in production.
PDF · 18 pages
Questions we are asked first.
Want this applied to your business?
Talk to us
Discuss retrieval architecture for production ai
Tell us where you are. We will point at the parts of this that matter for your situation and say plainly what we would do first.
Talk to us