How to Pilot AI With Company Documents Safely

How to Pilot a Source-Grounded Assistant With Company Documents

Begin with one approved, low-consequence question and a small authoritative source set. Define access, product and account, citation review, failure tests, retention, deletion, and the accountable owner before any company document enters the system.

The pilot is successful only if it helps a qualified reviewer reach accepted work with less total effort and no unacceptable failure. Fluent answers, high usage, or impressive demos are not enough.

flowchart TD
    A["Authorized task and users"] --> B["Approved product, account, and data"]
    B --> C["Versioned source set"]
    C --> D["Representative test cases"]
    D --> E["Citation and answer review"]
    E --> F{"Accepted with no unacceptable failure?"}
    F -->|"Yes"| G["Constrain or expand with approval"]
    F -->|"No"| H["Correct, rollback, or stop"]

1. Name the job

Choose a task such as locating a passage in a manual, comparing two approved procedures, preparing a study guide, or summarizing a bounded set of meeting notes.

State the users, question types, expected output, baseline process, and decisions that remain outside the assistant. “Understand the company” is not a task. It hides scope and makes failure difficult to measure.

2. Establish authority before access

Identify who owns the documents and who may use them for this purpose. Classify confidential, personal, regulated, privileged, licensed, copyrighted, customer, employment, security-sensitive, export-controlled, and contract-restricted material.

The pilot should not proceed because one employee can open a file. Access to a document does not automatically include permission to copy it into another system, expose it to another vendor, use it for model feedback, or generate a new derivative.

3. Verify the exact product path

Record the product, account, plan, tenant, administrator, integrations, regions, model path where known, retention, feedback behavior, human review, training or improvement use, sharing, logging, export, deletion, and support route.

Google's current Gemini Notebook privacy and terms page distinguishes account contexts and feedback handling. The same notebook can also appear in Gemini Apps, where grounding and activity behavior differ. A product name alone is not a sufficient data boundary.

4. Build a source register

For every included document, record title, owner, version, effective date, approval state, access class, scope, source of truth, and replacement or retirement rule.

Remove superseded duplicates when possible. If historical versions are required, label them so the assistant and reviewer can distinguish a current instruction from an old record.

State what is absent. A notebook built from one region's procedures should not answer as if it controls every region.

5. Create a representative test set

Start with questions whose correct disposition is already known. Include ordinary answers, absent answers, conflicting documents, exceptions, tables, footnotes, scans, images, ambiguous terms, and requests that should be refused or escalated.

Record the expected source and the acceptance rule before running the test. Otherwise, a plausible response can quietly redefine success.

6. Test retrieval before synthesis

Ask the system to locate the relevant source and passage before requesting a summary or recommendation. Check whether it selected the right document, found the controlling language, and exposed enough context to understand definitions and exceptions.

Current Gemini Notebook help says chat can use selected notebook sources and show direct source material as citations. That behavior makes inspection easier. It does not prove the passage is controlling or correctly interpreted.

7. Separate document text from interpretation

The output should distinguish what a document says from the assistant's summary, comparison, or inference. A generated interpretation should never appear as if it were quoted policy.

The qualified role that already owns the decision keeps authority. The assistant can find an underwriting guideline without becoming an underwriter. It can surface a compliance passage without granting legal approval.

8. Record failures and corrections

Preserve unsupported answers, weak citations, missed sources, version confusion, access errors, inappropriate disclosure, slow responses, questions that should have been declined, and reviewer corrections.

Do not repair an answer manually and then mark the assistant correct. The failure record is the evidence needed to improve the source set, instructions, permissions, and process.

9. Measure the complete workflow

Compare the assisted process with the baseline using accepted answers, citation fidelity, coverage, reviewer time, correction time, unacceptable failures, incident effort, maintenance effort, and total cost.

Count source preparation, upload, synchronization, administration, user support, review, rework, and deletion. Generation time alone is not productivity.

10. Decide whether to expand, constrain, or stop

Expansion should name the next task, users, documents, data class, controls, reviewer capacity, measures, and stop conditions. A successful low-risk lookup pilot does not authorize high-consequence decisions or a wider source set.

Stop or roll back when the system crosses a data boundary, exposes unauthorized information, misses controlling sources, produces unacceptable interpretations, cannot be corrected reliably, or creates more total work than the baseline.

Maintain the knowledge system

Assign owners for adding, updating, and retiring sources. Re-test critical questions after changes to the model, product, account, integration, source collection, policy, or workflow.

If the product stores copies instead of live references, define a synchronization process. A grounded answer can be wrong because it is grounded in an old version.

The underlying concept from E017 remains useful: selected company documents can make an assistant easier to inspect. The operating lesson is that source authority, data permission, review, correction, and exit have to be designed with the retrieval experience.

For the conceptual model, read [[Source Grounding Makes AI Answers Reviewable Not Correct]]. E019 helps select a first workplace AI task. E042 provides the wider bounded-pilot system, and E097 helps decide whether the task needs manual work, a workflow, or an agent.

This guide was checked on July 28, 2026 against current Gemini Notebook source, chat, account, and privacy documentation and NIST's AI Risk Management Framework. It is a general operating method, not authorization to process any specific company data.

AI assisted with research organization, structure, drafting, and validation. Dalton Anderson remains the attributed author and final editorial authority. The transcript and linked public sources control factual claims. Publication remains unauthorized.