A chatbot on your knowledge base (RAG)

A new employee asks, “How do we issue a corrective invoice?” and gets the procedure with a link to the document, instead of digging through a folder of 200 files. That is one way to use an internal knowledge base with a RAG chatbot. The same approach can give customers answers from your terms, manuals or price lists.

Making it work takes attention to the documents, the search and the way answers are checked. Uploading a set of files does not establish that the bot will use them correctly.

What happens when someone asks a question

RAG combines retrieval from your materials with answer generation. The system first searches for passages related to the question. It then passes them to a model, which uses them to put together a response. This is the basic approach described in Microsoft’s documentation on RAG.

Suppose a customer asks about delivery charges. The system finds the relevant part of the price list, and the model explains the conditions and links to the source. The customer can then open the document behind the answer.

Preparing this kind of knowledge base does not, by itself, mean training the model on your files. The model receives the retrieved passages when handling a question. What the provider does with the data you send, and how long it keeps that data, needs to be established separately.

Which questions this can help with

A useful starting point is questions whose answers are already written down somewhere but are hard to find. An employee needs to know where to report an issue to a supplier. A salesperson checks whether a service includes installation. A customer needs instructions for replacing a part.

In each case, you can identify a document and check the answer against it. Search that takes meaning into account can also help: someone might ask about “cancelling an order” while the instructions use a different phrase. You still need to check that the system found the right procedure and did not confuse two similar situations.

If the question concerns the status of a particular order, an instruction manual will not be enough. The bot needs access to current information in the order system. That connection is an additional part of the project.

What the knowledge base will not sort out for you

It cannot reliably resolve conflicting documents without rules for choosing a source. If the knowledge base contains three price lists and nobody has marked which one applies, the system may pick an old price. You need to identify the versions, remove outdated files or specify when each version applies.

The same applies to information nobody has written down. A question about an exception usually agreed over the phone should produce “I don’t have that information in the documents” and a way to contact someone who can help. That is the right behaviour, although someone expecting an answer to every question may be disappointed.

RAG does not eliminate the risk of made-up answers. Search can miss an important passage, and the model can misread a condition or add something of its own. A link to a document does not prove the answer is correct either. Testing needs to include opening the source and comparing it with what the bot wrote.

Retrieval alone is also a poor fit for adding up all the amounts on thirty invoices. It may find only some of the documents. That task needs a complete set of data and calculations in a spreadsheet or code; a chatbot can provide a way to request the work.

How to check whether the answers are useful

Collect questions people actually ask in your company. For each one, write down the expected answer and the document that supports it. Include a few questions with no answer in the knowledge base, and some where two products or versions of a service could be confused.

For each test, check:

whether the system found the right document;
whether the answer includes the conditions and exceptions in the source;
whether the bot can report missing information;
whether the user receives only content they are allowed to access.

The last point matters particularly inside a company. An invoicing procedure and a salary document might be on the same drive, but they should not be available to the same people. Permissions need to apply at retrieval time, as explained in the documentation on RAG access controls.

If answers are intended for customers, you can start by giving suggestions to an employee. They receive a draft with its source, check it and make corrections. This shows which questions the bot helps with and which create more work. Any later move to direct replies depends on those test results and the consequences of an error.

Who will keep the documents up to date

A price change needs to reach the materials the bot uses. Depending on the setup, that might mean replacing a file, importing the data again or running an automatic sync. Agree who does this and how they check that the new version is available.

Costs need the same attention: document preparation, retrieval, model usage, integrations and maintenance all contribute. I explain those expenses in the post on AI implementation costs.

For a first trial, choose one area, such as delivery questions. Gather the current documents and questions from customers. The test will also show which answers are missing from those documents.