How to Ground an AI Assistant in a Knowledge Base

How to Ground an Assistant in a Curated Knowledge Base

Ground an AI assistant by treating its knowledge base as a maintained evidence system. Approve each source, preserve provenance, enforce access before retrieval, test retrieval and answer use separately, require citations or refusal, and support refresh, correction, and deletion.

Uploading files is only the beginning.

flowchart LR
    A["Authorize sources"] --> B["Preserve provenance"]
    B --> C["Parse and index"]
    C --> D["Retrieve with access controls"]
    D --> E["Generate with citations"]
    E --> F["Evaluate retrieval and answer"]
    F --> G["Monitor, refresh, correct, delete"]
    G --> B

Start with a source manifest

Create one record for every approved source. Include the owner, title, version, effective date, rights basis, audience, sensitivity, source location, canonical identifier, ingestion date, refresh trigger, and deletion route.

Do not assume that access to a file grants the right to republish its contents through an assistant.

Separate public, internal, confidential, personal, licensed, and restricted material. Record whether the assistant may quote, paraphrase, summarize, or only use a source for internal reasoning.

When a source contains information about customers, employees, guests, or other people, privacy and consent questions remain even if the organization owns the storage account.

Preserve the original and its provenance

Keep an immutable source object or a verifiable link to the authoritative record. Record a hash where practical.

Derived text should point back to the exact object, page, section, timestamp, or record that produced it. Parsing, optical character recognition, transcription, cleanup, and translation can each introduce errors.

The lineage should make it possible to answer: "Which source supported this sentence, what version was used, and can we remove it?"

Design parsing and chunking for the material

Chunking should follow the structure and retrieval needs of the source. A policy may need section-aware chunks. A podcast transcript may need speaker and timestamp boundaries. A product catalog may need one record per item and variant.

Avoid splitting definitions from their qualifications, tables from their headers, or claims from their dates.

Store stable source identifiers and locations with every chunk. If the system cannot reconstruct a useful citation, its provenance model is incomplete.

Google's prompt design guidance shows how supplied context can make an answer more specific to the material. That prompting pattern does not establish source rights, retrieval quality, access control, or citation fidelity.

Enforce access before retrieval

The retriever should only search material the user is authorized to access. Filtering after generation is too late.

Apply identity, tenant, role, purpose, and record-level controls outside the language model. Minimize what enters the prompt. Avoid sending secrets or unnecessary personal data.

Log retrieval decisions without creating a new privacy problem. Retention, access to logs, deletion, and incident response need explicit owners.

Test retrieval separately from the answer

Create questions with known supporting passages. Measure whether the relevant source appears, whether irrelevant material dominates, and whether permissions hold.

Then evaluate the answer. Check whether it uses the retrieved evidence faithfully, distinguishes fact from inference, respects dates and scope, cites the right source, and refuses when support is absent.

A correct passage in context does not guarantee a correct answer. A fluent answer does not prove that the right passage was retrieved.

The NIST Generative AI Profile identifies risks across generative AI systems and value chains. It is useful for placing retrieval inside broader governance rather than treating it as a factuality switch.

Treat retrieved text as untrusted input

Documents can contain malicious or irrelevant instructions. A public webpage, uploaded file, support ticket, or email can try to override the assistant's rules.

OWASP's prompt injection guidance explains why indirect instructions in external content can alter model behavior.

Separate trusted system instructions from retrieved content. Limit tools and permissions. Validate actions outside the model. Do not let a retrieved sentence authorize data access, a purchase, a message, or code execution.

Make citations useful

A citation should let the reader inspect the supporting material. Use a stable source name, version or date, and a direct location when available.

Do not attach a citation to a paragraph if the source supports only one sentence. Do not cite the knowledge base as a whole.

If the assistant cannot identify adequate support, it should say what is missing and offer an allowed next step. "I do not have an approved source for that" is often more useful than a plausible guess.

Operate the lifecycle

Define how sources are added, reviewed, superseded, corrected, and removed. Reindexing must not silently revive deleted or expired material.

Monitor unanswered questions, weak retrieval, unsupported claims, stale citations, permission failures, user corrections, and source changes. Preserve enough version information to reproduce material incidents.

Changes to the source, parser, chunking, embeddings, index, retrieval logic, prompt, model, permission system, or answer policy require proportionate reevaluation.

Episode 28 imagined a Venture Step assistant that could make unstructured podcast knowledge easier to query. The durable implementation is not "upload every transcript." It is a source manifest, canonical episode identities, preserved raw transcripts, evaluated retrieval, visible citations, and deletion that actually propagates.

For a parallel evidence discipline, read [[How to Read Training Data Disclosures Without Inventing the Dataset]]. For the assistant contract that comes before retrieval, use [[How to Design a Narrow AI Assistant]].

This guide was developed with AI assistance from E028, NIST, OWASP, and the linked knowledge-lifecycle framework. Dalton Anderson remains the author. Technical, data-rights, privacy, security, domain, accessibility, source, and founder review are required before publication or deployment. Publication and deployment are not authorized.