Understanding Retrieval Augmented Generation (RAG) and Grounding and Their Inter-related Implementation

RAG (Retrieval Augmented Generation) and Grounding are related concepts but technically very different. They are often used together in the implementation of Large Language Models (LLMs). Grounding ensures AI’s answer is verifiable with real world domain specific evidence rather than just AI model’s internal knowledge. RAG is the process of fetching external information, this can be from the web, databases or even private collection of documents. Lets dig deeper.

Itika SinghalOctober 25, 20262 min read

Understanding Retrieval Augmented Generation (RAG) and Grounding and Their Inter-related Implementation

What is Grounding?

The act or goal of making AI’s answer factual and verifiable with evidence set in web URLs is called grounding. The aim of grounding is that AI should not rely solely on the data it is trained upon, which can become outdated leading to hallucinations. The URLs cited with grounding provide context to what AI says in its answer.

This technique is useful in research based work where claims can be verified against the evidence present.

What is Retrieval Augmented generation (RAG)?

RAG is a framework which enables fetching of information dynamically from external knowledge bases before generating a response. RAG is used to achieve grounding at scale.

RAG follows a Retrieve - Augment - Generate mechanism. A search system retrieves information from external databases, keyword search or hybrid APIs based on user’s prompt. The retrieved chunks of data along with evidence are sent to LLM as context. LLM then generates a final response using the retrieved facts.

What is a grounded answer? How are RAG and Grounding implemented together?

RAG is a method to achieve grounding. External information and evidence may come from web pages, documents, databases, knowledge bases etc. Through Retrieval augmented generation method, information along with evidence is retrieved. An answer is synthesised which has verifiable evidence along with it. This is called the grounded answer.

Scenarios when RAG and grounding are implemented:

When RAG and grounding are implemented together, they are three possibilities that can happen:

RAG implementation without effective grounding: System may retrieve a document but LLMs may interpret is wrong and / or make unsupported claims. This can often lead to hallucination.
Grounding without RAG implementation: A user may upload a document within the LLM while asking a question. LLM may then base its answer completely on that document without a separate retrieval step.
RAG with effective grounding: Best scenario of the three. The LLM will implement RAG, retrieve relevant, reliable information and generate an accurate response to the user’s prompt along with evidence that supports it.