AI Agents & RAG
AI Knowledge Base Assistant
A RAG-powered chatbot that answers questions from a company's own documentation, with retrieval and prompt design tuned for accurate, context-grounded responses.
Overview
Built an AI chatbot powered by a structured knowledge base capable of answering questions using company documentation. Designed retrieval workflows, prompt engineering, and context-aware responses to reduce manual customer support and improve information accessibility.
Problem
Answers to common questions already existed somewhere in scattered documentation, but finding them meant searching multiple documents manually or asking a person who happened to know — slow for users and repetitive for whoever kept answering the same questions.
Business Goal
Let anyone ask a question in plain language and get an accurate answer grounded in the actual documentation, instead of a generic LLM guess or a manual search.
Architecture
System design
Documentation is chunked and embedded into a Supabase vector store. Incoming questions are embedded and matched against that store to retrieve the most relevant passages, which are then passed as grounded context to Claude/OpenAI along with a prompt designed to answer only from the retrieved material and say so when it can't.
Challenges
What made this hard
An LLM will confidently answer even when the retrieved context doesn't actually contain the answer.
Constrained the system prompt to answer strictly from retrieved passages and explicitly say when the knowledge base doesn't cover a question, rather than falling back on the model's general knowledge.
Chunking documentation naively breaks answers across chunk boundaries.
Tuned chunk size and overlap against real documentation structure so retrieved passages stay coherent instead of cutting answers in half.
Business Impact
What it changed
Reduced manual, repetitive support questions by giving users a direct, self-serve path to accurate answers grounded in existing documentation.
Lessons Learned
- Retrieval quality matters more than model choice — a better retriever with a mid-tier model consistently beat a great model on bad retrieval.
- Explicitly designing for 'I don't know' responses builds more trust than an assistant that always sounds confident.
Future Improvements
- Add source citations so users can verify answers against the original document.
- Feed unanswered/low-confidence questions back into a documentation-gap report.
Related