Status: An anonymized target architecture for a pilot. This is not a claim that a production system is deployed or that it has achieved measured outcomes.
Insurance RAG should start with a question more basic than which model should we use: who is allowed to see which source, for what decision, and how will they verify the answer?
For an insurer, an underwriter looking for policy guidance and an agent or customer looking for product information are not two modes of the same assistant. They are two products with different source approval, permissions, failure modes, and release obligations. Treating the public experience as a filtered view of internal knowledge is how an apparently helpful system becomes a data-boundary incident.
This is the target architecture I would validate in an anonymized pilot: an internal evidence system for underwriters first, with a separately governed path for a future customer-facing FAQ.
The boundary comes before the chatbot
The internal pilot has a narrow job: help an underwriter find current policy guidance, explain it in plain language, and cite the originating document and section. It does not approve coverage, adjudicate a claim, change a policy record, or turn an ambiguous question into a confident decision. Those remain human responsibilities.
The public FAQ has a different job: explain approved product information and send the person to a human, agent, or established workflow when the answer is personal, regulated, incomplete, or outside the approved corpus. It cannot inherit material from claims, underwriting, internal policy discussions, pricing operations, or employee guidance.
The two products may reuse infrastructure patterns. They must not share a single corpus with a UI toggle. Each needs its own source owner, approval path, access model, test set, and release cadence.
Classify before you chunk
The hardest RAG work starts before embedding. Every source needs an owner, effective date, document type, audience, retention treatment, sensitivity label, and access rule. If a policy has been superseded, the system must either retrieve the current version or say that it cannot establish the answer—not make the old wording sound current.
The extraction path should also fit the source. A long policy PDF can be segmented around headings, endorsements, sections, and effective dates. A spreadsheet is different: tables may carry the very relationship that gives a rule meaning. Flattening it into paragraphs can remove headers, units, formulas, and row context. That needs its own structure-preserving extraction and evaluation path.
This is why putting every document in a vector database is not an architecture. It is an ingestion event.
Retrieve only what the person may see
Authorization belongs in the retrieval path, before the model receives a chunk. The application should establish the caller’s identity and role, request only sources that identity can access, and keep the returned evidence tied to the answer. Prompt instructions are useful guardrails; they are not access control.
Azure is the selected pilot stack. Azure AI Search provides the retrieval layer because it supports document-level access patterns and hybrid retrieval; Azure Document Intelligence provides a managed extraction option for the PDF and spreadsheet estate; Entra-backed identity carries the caller context; and an approved Azure-hosted model service produces only grounded answers. The team still owns the security architecture: source permissions, metadata quality, token handling, failure behaviour, and verification. Preview-only permission features should not become an unexamined production dependency.
For the pilot, I would use hybrid retrieval: exact keyword matching alongside vector similarity, then evaluate whether reranking earns its additional latency and cost. Insurance questions often contain policy terms, endorsement names, form identifiers, and exclusions where exact matching matters as much as semantic similarity.
Make the answer auditable—or decline to answer
The answer contract should be boring on purpose: answer only from authorized retrieved sources; cite the source document, effective date, and section for each material statement; state when the material is insufficient, conflicting, or stale; and route consequential questions to the existing human workflow.
Abstention is not a product failure. A fluent but unsupported answer is. The pilot should measure retrieval quality separately from writing quality: first, did it find the right authorized source? Second, given that source, did it faithfully explain it? Combining those two failures into one AI accuracy number makes the eventual fix harder.
Start with an evaluation set, not a demo
Before expanding the corpus, create a representative question set with underwriting subject-matter experts. It should include common questions, rare policy language, conflicting or superseded documents, out-of-corpus requests, and attempts to retrieve information from another role’s domain.
The baseline is the existing workflow: how an underwriter finds and checks the same guidance today. The pilot then tracks retrieval recall, citation coverage, human acceptance, appropriate abstention, unauthorized-result attempts, latency, freshness, and cost per answered question. It also needs a named owner for source updates and removals; an index without an owner becomes a stale-policy machine.
Why I would not start with an agent
An agent adds planning, tool selection, and sometimes write access. None of those are necessary for the first job: find approved evidence and explain it. A constrained retrieval service is simpler to evaluate, less expensive to operate, and easier to audit.
If evaluation later shows that users need a multi-step workflow—such as comparing several authorized policy forms, gathering a structured review packet, and submitting it for human approval—then add bounded orchestration. The system should earn additional autonomy with evidence, not receive it because agentic is in the roadmap.
This follows the broader principle behind enterprise context engineering: retrieval quality, authority, freshness, and ownership determine whether an AI answer can be trusted. The same is true of governed AI operating contracts: specify the value, authority, evidence, and stop conditions before granting a system more reach.
Technology should explain a constraint
I would mention technology in the implementation appendix, not lead with it. The main story is the decision system: source classification, separated product boundaries, identity-aware retrieval, evidence-based answers, evaluation, and human accountability.
The selected pilot stack is Azure AI Search for retrieval, Document Intelligence or a structure-preserving parser for extraction, Entra-backed identity, secure storage, and an approved Azure-hosted model service. Exact models, regions, index settings, data schemas, and security controls do not belong in a public architecture note.
Provider terms do not eliminate the team’s responsibility for data minimization, residency, retention, and security review. They are one input to the decision—not a substitute for it.
The next product is not a shortcut
The customer-facing FAQ is the next product relationship for Customer Needs Analysis. It should be built from an independently curated public corpus, with an explicit answer boundary and a safe handoff to agents or customer support. It is not the internal underwriting assistant with different branding.
That separation is slower at the beginning. It is much cheaper than trying to prove, after launch, that a public answer was never influenced by an internal document.
The next useful milestone is not a broader chat interface. It is a tightly scoped underwriting pilot with a known corpus, named users, a representative evaluation set, and a clear failure path. When that produces evidence, the architecture note can become a real pilot report. When the customer FAQ is ready and its project page is live, it can become the second product story in the same series.
The final commercial question is whether this creates value beyond time saved. That is the measurement discipline covered in How to Measure AI ROI Beyond Hours Saved.
Questions architecture leaders should settle before a pilot
Should the pilot start with an insurance agent FAQ or an underwriter assistant?
Start with the narrower, internal underwriting guidance workflow if the organization can define the permitted corpus and evaluation set. It exercises the hard controls first. Build the public FAQ separately from approved public sources once its product boundary is ready.
Is hybrid retrieval enough for the first version?
It is a defensible starting point because exact policy terms and semantic questions both matter. Whether reranking or agentic retrieval is justified should be decided with representative evaluation data, not assumed from a demo.
What is the most important pilot metric?
There is no single metric. Track whether the right authorized evidence was retrieved, whether the answer remained faithful to it, whether the system abstained correctly, and whether a user could complete the guidance task more reliably than in the existing workflow.
Sources
Primary documentation and standards used to inform the target architecture.
