RAG beats a clever chatbot — if your documents are actually maintained

Most companies now have a chatbot. Few have a source of truth the chatbot is allowed to read. The model then improvises policy, prices, and clinical advice. That is not an LLM failure. That is an information-architecture failure.

RAG — retrieval-augmented generation — only works if retrieval can find a passage that answers the question. That means documents with titles, dates, owners, and a kill date. A 2019 PDF sitting next to a 2026 pricing sheet will be retrieved at random unless you filter on metadata.

Start with the ten questions your team already asks in Slack. Write canonical answers. Put them in a tool humans will edit (Notion, Git, a CMS). Chunk on headings, not on an arbitrary token window that splits a table in half. Store product line, locale, and audience as metadata.

Evaluations are not optional. Your operations lead should write thirty real questions with cited answers. You run them every time you change chunking or the model. “It feels better” is not a release gate.

Citations should be visible to the user or stored for audit. If the model cannot point at a passage, it should refuse and open a ticket. Containment rate without an accuracy number is how companies ship confident nonsense.

Finally, budget the unglamorous work: who uploads a new SOP, who retires a dead one, what happens when two documents disagree. AutoFy Tech will not install a vector database on top of a folder nobody owns. We will help you name the owner first.