RAG is one of the easiest AI applications to deploy. It addresses a straightforward problem: models don’t know your internal materials and shouldn't answer from thin air. Instead, they first retrieve relevant passages from a knowledge base before generating answers based on those excerpts. Once implemented, users can ask questions like "What changed in last quarter's channel policies?", "What are the payment milestones in this contract?", or "How does the new employee reimbursement process work?" The system provides an answer along with its cited sources.
Who This Is For
This tutorial is designed for enterprise knowledge base managers, developers, operations teams, and consulting groups. If your materials are scattered across PDFs, web pages, Feishu/Notion docs, customer service FAQs, or product manuals, you can use RAG to improve retrieval efficiency. It is not suitable for handling highly real-time data such as inventory levels, orders, or account balances; those scenarios should prioritize direct calls to databases or business APIs.
Step 1: Organize Document Sources
Do not immediately dump all files into the vector database. First, create an inventory of sources detailing document type, owner, update frequency, permission level, and whether access to the AI system is permitted. Common internal enterprise documents fall into four categories: policies and procedures, product manuals, training materials, and project archives. Each category has its own update cadence and distinct permissions.
It is advisable to start with a well-defined collection, such as a "Customer Support Knowledge Base" or "Employee Administrative Policies," keeping the document count between 50 and 300 items. A scope that is too broad will make troubleshooting an agonizing process.
Step 2: Cleaning and Chunking
RAG quality depends heavily on chunking strategies. Common issues in PDFs include repeated headers and footers, misaligned tables, awkward line breaks splitting sentences, and OCR errors from scanned documents. For web pages, typical problems involve navigation bars, advertisements, and related recommendations being mixed into the main content. During the cleaning phase, strive to preserve heading hierarchies, paragraph structures, and table semantics.
For chunking, use "semantic blocks" rather than fixed character counts. A single policy clause, a FAQ entry, or a product feature description should each constitute one block. In Chinese contexts, keep chunks between 300 and 800 characters; too short risks losing context, while too long degrades retrieval precision. Every chunk must include metadata: title, source URL, publication date, document category, and permission tags.
Step 3: Vectorization and Indexing
Vectorization converts text chunks into numerical representations suitable for retrieval. You can use embedding models from OpenAI, Cohere, Voyage, BGE, or Jina. For Chinese knowledge bases, it is crucial to specifically test recall performance on Chinese queries rather than relying solely on English leaderboards. Suitable vector databases include pgvector, Qdrant, Milvus, Pinecone, and Weaviate.
For a minimal setup, we recommend PostgreSQL with the pgvector extension; it offers simple deployment and straightforward management of permissions and backups. If you are dealing with massive datasets or require high-concurrency retrieval, then consider deploying a dedicated vector database.
Step 4: Retrieval, Reranking, and Filtering
After a user submits a query, the system first vectorizes the question to retrieve the top 10 to 20 most relevant fragments. Next, it employs a reranking model or rule-based logic to prioritize truly relevant segments at the top of the list. In enterprise scenarios, permission filtering is also essential: documents that a user lacks access to must be excluded from the context window, even if they are semantically relevant.
Here is a practical tip: do not rely solely on vector search. Hybrid retrieval—combining vector search with keyword matching—is generally more robust. For instance, if a user asks about "invoice headers for travel expense reimbursement," the keywords "invoice header" are critical; pure vector search might retrieve general information about "reimbursement processes" while missing specific clauses related to invoice details.
Step 5: Prompts and Citations
When generating answers, the prompt must be explicit: "Answer only based on the provided materials; if information is insufficient, state that you do not know; every key conclusion must cite its source." Ideally, citations should map to document titles and specific paragraphs rather than just listing URLs. When a user clicks a citation, they should jump directly to the relevant location in the original text.
A common mistake is treating citations as decoration: appending a few sources after writing an answer without integrating them properly. The correct approach ensures that every sentence in the answer can be traced back to its retrieved snippet. This not only enhances credibility but also helps users identify issues where knowledge base content may be outdated or missing.
Running a Minimal RAG with LlamaIndex
Below is the minimal viable approach: load files from docs/, build an index, then ask questions. Production environments will require adding permissions, incremental updates, and citation display; for local validation, simply get this step working first.
mkdir rag-demo
cd rag-demo
npm init -y
npm install llamaindex
mkdir docsimport { VectorStoreIndex, SimpleDirectoryReader } from "llamaindex";
async function main() {
const documents = await new SimpleDirectoryReader().loadData({
directoryPath: "./docs",
});
const index = await VectorStoreIndex.fromDocuments(documents);
const queryEngine = index.asQueryEngine();
const answer = await queryEngine.query({
query: "What are the invoice title requirements in the reimbursement process?",
});
console.log(answer.toString());
}
main();Recommended Project Directory Structure
rag-demo/
docs/ # source documents
scripts/ingest.js # document cleaning and ingestion
scripts/evaluate.js # test set evaluation
src/search.js # retrieval and reranking
src/answer.js # answer generation with citations
data/testset.json # test set of real questionsWhat Interface Validation Screens Should Look Like
A RAG tutorial must include at least three verifiable interface screenshots: the document import page, the retrieval results page, and an answer page with citations. The retrieval results page must display chunk titles, source documents, update timestamps, and ranking scores; the answer page must show citation numbers that, when clicked, jump directly to the corresponding original text passages. Capturing only a chat window showing "AI answered correctly" is meaningless because readers cannot see the recall process.
Common Pitfalls
| Symptom | Where to Check First | Fix |
|---|---|---|
| The system claims it doesn't know despite documentation existing | Top 5 retrieval results | Adjust chunking; add titles and synonym keywords to chunks |
| Answers cite outdated policies | updatedAt in document metadata | Filter by version during retrieval; mark old documents as archived |
| Different departments see the same answer | Whether permission filtering runs before recall | Filter documents by user permissions first, then perform vector recall |
| Answers are fluent but unsupported by sources | Generation prompts and citation mapping | Do not display sentences without citations, or label them "Insufficient Data" |
| Table content is frequently answered incorrectly | PDF table parsing results | Convert tables separately to Markdown or CSV; do not mix with main text during chunking |
Minimum permission filtering can be implemented as follows:
function filterByPermission(chunks, user) {
return chunks.filter((chunk) => {
if (chunk.visibility === "public") return true;
return chunk.allowedDepartments?.includes(user.department);
});
}Pre-Launch Checklist
Conduct at least three rounds of checks before launch. Round one audits the documents: verify that each document has a title, source, update time, owner, and permission tags; confirm expired documents are taken offline; ensure no single policy exists in multiple versions simultaneously. Round two tests retrieval: prepare 50 real-world questions to confirm correct snippets appear within the top five recall results; if they do not, first refine chunking and keywords before rushing to switch models. Round three evaluates generation: check whether answers cite sources, refuse to answer when data is insufficient, and avoid conflating "suggestions," "regulations," and "experiences" into a single tone.
For enterprise teams, add an operational mechanism: designate who updates the knowledge base, how often reviews occur, and where users should report incorrect answers. A RAG system without maintenance may seem intelligent at launch but will devolve within three months into a machine that diligently cites outdated information.
Alternative Approaches
If the available data is minimal, it may be simpler to place documents directly into the model's context. For queries requiring real-time information, database lookups or business APIs are preferable. If your team lacks engineering resources, you can first validate requirements using existing tools such as Dify, LlamaIndex Cloud, OpenAI File Search, Notion AI, and Feishu Knowledge Q&A.
Summary
The core of RAG is not merely "connecting a vector database," but building a reliable information chain: trustworthy documents, logical chunking, accurate retrieval, clear permissions, and citable answers. Only by solidifying this chain can RAG evolve from a demonstration project into a knowledge entry point that teams are willing to use every day.