GraphRAG is a retrieval approach that adds a knowledge graph layer on top of RAG. Ordinary RAG chops documents into passages, turns them into vectors, and hands the most semantically similar ones to the model. GraphRAG adds a step: a model first extracts the entities (people, companies, products, concepts) and relationships (acquired, depends on, reports to) from the documents into a graph, and retrieval can then walk that graph hop by hop.
It targets a specific failure mode of ordinary RAG: the answer exists in no single passage, but is spread across several documents and only holds once they are connected.
Grab It in One Sentence First
Ordinary RAG finds the passages most similar to your question; GraphRAG finds the network of relationships around it.
An analogy. Ordinary RAG is keyword search across a pile of files, surfacing the few most on-topic pages. GraphRAG first draws a relationship map and timeline over that pile, so when you ask "how is Zhang connected to this project's delay," it can walk "Zhang → owns module A → module A depends on supplier B → supplier B delivered late" — even though those four facts sit in four separate documents and no single passage mentions both Zhang and the delay.
How It's Built
The graph-building phase is offline and the expensive part: a model reads every chunk, extracts entities and relations, then resolves them (merging "Apple Inc.", "Apple", and "AAPL" into one node). At document-collection scale, the token cost is substantial.
Community summarization is the key design in Microsoft's version of GraphRAG. After the graph is built, community detection groups tightly connected nodes into thematic clusters, and each cluster gets a summary. That way, global questions such as "what are the main disputes across this corpus" — which no top-K passage retrieval can answer — can be handled by rolling those summaries up.
The query phase typically has two modes. Local search locates the entities in the question, expands one or two hops along edges, and passes both the neighboring nodes and the underlying passages to the model. Global search goes through community summaries to answer summarizing questions.
Where It Beats Plain RAG, and Where It Doesn't
Strong on multi-hop and global questions. Questions requiring cross-document chaining ("which of company A's suppliers are also company B's customers") and questions requiring overall synthesis ("what common problem do these five hundred tickets reveal") are structurally out of reach for plain RAG — the first has no passage containing the full chain, and the second can only ever see a local slice through top-K retrieval.
Weak on cost and freshness. Building the graph takes a great many model calls, and incremental maintenance as documents change is awkward. Extraction itself errs: a botched entity resolution or an inverted relation leaves a false fact in the graph, and that error is more insidious than retrieving an irrelevant passage — the model treats it as confirmed structured fact.
Hybrid usually wins. In practice the best-performing setups combine vector retrieval, keyword retrieval (BM25), and graph traversal, weighting them by question type rather than relying on any one. Some implementations avoid model calls at retrieval time entirely, doing only indexing and traversal to keep latency down.
Versus Neighboring Concepts
Versus knowledge graphs. A knowledge graph is the data structure itself, built by hand or by extraction, and useful well beyond question answering. GraphRAG is one application of that structure to retrieval-augmented generation — the graph is a means, not the end.
Versus RAG. GraphRAG is a variant, not a replacement. In most settings, better chunking, hybrid retrieval, and reranking deliver more benefit than a graph, at far lower cost.
Versus agent memory. Time-aware graph memory is essentially the same technology aimed at a different question — memory cares when a fact was valid, GraphRAG cares how entities connect. Implementations often share a graph database.
Where People Get It Wrong
"A graph eliminates hallucination." It doesn't. A graph supplies a fuller evidence chain, but extraction errors become false facts in the graph, and the model will produce a fluent answer on a wrong premise all the same. Evals and visible source attribution are still required.
"Every RAG system should upgrade to GraphRAG." Most shouldn't. If your questions are single-hop fact lookups — "what does the document say about X" — the graph adds nearly nothing while multiplying cost. Confirm you actually have multi-hop or global-summary needs first.
"Build the graph once and you're done." Documents change, so the graph must change. Incremental updates, entity merging, and pruning stale relations are routinely underestimated, and are why many GraphRAG projects stall after launch.
When It's Worth It
It pays off when three signals coincide: questions need cross-document chaining (multi-hop); the corpus is dense in entities and relations (real structure between people, organizations, products, events); and documents don't change often (or maintenance eats the benefit).
Good fits are due-diligence material, case files, incident-ticket archives, research literature reviews, and internal org-and-system dependency maps. Poor fits are FAQ collections, product manuals, and policy texts, where one passage answers the question — there, getting basic RAG's chunking and reranking right usually works better.