Before you build an AI knowledge base, organise the answers.

Start with recurring questions, not every company folder

Ask one team to collect the questions that repeatedly interrupt its work. A design studio might need project-opening checklists, approved briefing templates and instructions for recording client decisions. A service business might need branch procedures and supplier onboarding steps. Choose a question set with a clear operational owner.

Then find the material a trusted colleague would use to answer those questions today. If the real answer exists only in somebody's memory, write and approve it first. If the process changes depending on an unspoken exception, document that exception. Connecting more folders will not resolve a disagreement about how the business actually works.

Understand what the assistant is doing with your documents

A common approach is retrieval-augmented generation, usually shortened to RAG. Microsoft describes this as searching for relevant source content and supplying it to a language model as context for an answer. This can connect an assistant to internal material without treating the model's general knowledge as the company handbook.

For the person using it, the important question is simpler: can I open the source and check the claim? Ask for a prototype that shows a short answer alongside the relevant document and passage. A source link is a checking aid, not a guarantee that every sentence in the response follows from that source.

The hard question is not whether AI can answer. It is whether the business can stand behind the source.

Give each source an owner and a usable status

Create a lightweight source register with the document title, responsible person, audience, current status and review date. Use meaningful states such as approved, draft and retired. File names like final-v3-new do not explain which instruction staff should follow. Where two sources conflict, the owner should resolve the disagreement before both enter the assistant's approved collection.

Keep genuinely different guidance separate. A Dubai branch and a London branch may use different handover hours or operational contacts. Add that context to the source itself and have the assistant ask which branch applies when the question is ambiguous. Do not let a familiar location name become a reason to guess.

Check permissions before the answer is assembled

Microsoft's RAG guidance recommends applying access control when source material is retrieved. This matters because hiding a document link after generating an answer does not remove information that has already appeared in the response. The assistant needs a clear relationship between the signed-in user and the sources they may use.

For a first staff-facing release, leave sensitive and unnecessary collections outside the scope. Test with separate synthetic accounts representing ordinary staff, managers and a user whose access has been removed. Ask the same questions from each account. Have the implementation team explain how permission changes affect the search collection and how that behaviour is verified.

Treat changed and deleted documents as acceptance tests

Ask what happens when a procedure is revised, moved or retired. A refreshed folder does not by itself prove that every searchable copy has been refreshed. For example, Microsoft documents that Azure Storage indexers detect changes, while deletion handling requires a separate strategy to avoid leaving old search documents behind. That is a product-specific example of a question every proposed implementation should answer.

Run a simple test: publish a synthetic procedure, ask about it, change an important instruction and ask again after the agreed update window. Then retire it and verify that the obsolete instruction is no longer offered as current guidance. Include failed updates in the operating checklist.

Test missing answers as carefully as correct ones

Microsoft's RAG design guidance separates retrieval evaluation from evaluation of the final response and recommends representative test material, including questions the sources do not cover. That distinction is useful for a small business: a wrong answer may begin with the wrong passage being found, or with a poor explanation of the right passage.

Create a review set containing ordinary questions, ambiguous requests, superseded guidance and questions with no approved answer. Ask a knowledgeable colleague to record the expected source and what a useful handover would say. Score unsupported details and incorrect citations separately from tone. A polite answer that invents an internal policy should not pass because it sounds helpful.

Make ownership part of the first release

The first version should have someone responsible for source quality and someone responsible for the running system. They may be the same person in a small business, but the two jobs should still be explicit. Give staff a visible way to report an outdated answer and show who receives that report.

Begin with a limited audience and a narrow source collection. Track questions that could not be answered, the effort needed to verify answers and the frequency of source corrections. Those observations tell you what to improve next. Expanding into another department should include its owner, access rules and test questions—not just another folder connection.

Put the idea to work.