RAG vs GraphRAG: when a knowledge graph is worth it
Plain RAG is enough for most question-answering over documents. GraphRAG earns its extra cost when answers depend on relationships between entities and multi-hop reasoning.
Start with plain RAG. It is cheaper, simpler to maintain, and handles most "find the right passage and answer from it" problems well. Move to GraphRAG when your questions depend on how entities relate to each other, or when a single answer requires connecting facts spread across many documents.
How plain RAG works
A retrieval-augmented system gives a large language model the right context at question time, instead of hoping the model memorized it. Four steps do most of the work.
- 01Chunksplit along headings
- 02Embedturn each chunk into a vector
- 03Retrievefetch the nearest chunks
- 04Rerankreorder by real relevance
- 05Answermodel cites the best few
| Step | What it does | Common trade-off |
|---|---|---|
| Chunking | Splits documents into prompt-sized pieces | Small chunks match precisely but lose context |
| Embeddings | Maps each chunk to a vector by meaning | Needs a vector database or extension |
| Retrieval | Finds chunks near the question | Add keyword search for part numbers and error codes |
| Reranking | Re-scores candidates against the question | Cheap, and often the best quality gain |
Splitting on natural boundaries such as headings and paragraphs usually beats cutting at a fixed character count. Chunks with similar meaning land near each other even when they share no keywords, which is what embeddings are for.
Done well, this reduces hallucination because the answer is anchored to retrieved text you can cite.
Where plain RAG fails
| Failure | Example question | Why retrieval misses |
|---|---|---|
| Multi-hop | Which suppliers provide parts in products that failed inspection? | Three records are needed, and no chunk holds all three |
| Entity relationships | Which contract governs which account? | The question is about links, not text similarity |
| Whole-corpus | What themes recur across all these reports? | Top-k returns a handful of chunks only |
| Ambiguous names | One person, three spellings | Chunks never get connected |
What a knowledge graph adds
A knowledge graph stores entities as nodes and relationships as edges: this contract belongs to that account, this person works on that project. GraphRAG builds that structure from your documents, usually by having an LLM extract entities and relationships during ingestion, then uses it at query time.
| Capability | What it gives you |
|---|---|
| Traversal | Follow edges to neighbors, which answers multi-hop questions |
| Entity grounding | Different mentions resolve to one node |
| Structure summaries | Related entities are grouped for whole-corpus questions |
| Explainable paths | You can show which relationships produced an answer |
Most production designs keep the vector search and add the graph beside it. Vectors find the starting points, and the graph expands from there.
What it costs
- Ingestion. Extracting entities means running an LLM over your corpus, and again when documents change.
- Schema and quality work. Someone decides which entity types matter and checks extraction. Entity resolution (deciding that two names are one thing) is hard.
- More moving parts. A second index to keep in sync, with more failure modes.
- Evaluation. You need real questions with known answers to prove the graph helps.
How to decide
- Collect 30 to 50 real questions users ask, and mark which need facts from more than one document.
- Build plain RAG with sensible chunking, hybrid search and a reranker. Measure it on those questions.
- Read the failures. Are they retrieval misses that tuning can fix, or answers that need relationship traversal?
- If most failures are relational or multi-hop, prototype a graph on a narrow slice of the data and compare.
- Keep the graph only if it beats the baseline by enough to justify the ingestion and upkeep cost.
Before you commit to a graph, check that:
- You have a written set of real questions with known answers
- Plain RAG uses hybrid search and a reranker, and has been measured
- The failures you read are relational or multi-hop, not retrieval misses
- Someone owns the entity schema and extraction quality
- You can afford re-ingestion when documents change
How this looks at Techdome
For a US conversational AI venture, we engineered a production multimodal conversational AI engine using n8n orchestration, LangChain and GraphRAG knowledge graphs. The stack keeps both retrieval layers inside PostgreSQL: pgvector for vector similarity and Apache AGE for the graph. It ran as a $10K commercial sprint delivered in 2 to 3 weeks and is in production.
Keeping vectors and graph in one PostgreSQL database means one system to back up, secure and operate rather than two. The same approach applies to our AI agents and voice AI work: ground the agent in your data first, and add structure only where the questions demand it.
Takeaways
- Start with plain RAG: good chunking, hybrid search and a reranker fix many problems that look like they need a graph.
- GraphRAG helps when answers depend on entity relationships, multi-hop reasoning or whole-corpus questions.
- The graph costs more to build and maintain: extraction, schema design, entity resolution and evaluation.
- Decide with a set of real questions and a measured baseline, not with a diagram.
- Keeping vectors and graph in one PostgreSQL database reduces operational overhead.
Have a project like this? Talk to our engineers.
