Retrieval-Augmented Generation: How AI Chooses What to Cite and How You Can Be Seen

Retrieval-Augmented Generation (RAG) is a critical framework that dictates which content Artificial Intelligence (AI) tools retrieve and subsequently cite before formulating an answer. Understanding RAG is essential for comprehending how platforms like ChatGPT, AI Mode, and other AI-powered search engines select the pages that form their responses. This guide delves into the mechanics of RAG in plain English, outlines the factors that increase content’s likelihood of being retrieved, and introduces Ahrefs Brand Radar as a tool to measure visibility within AI systems employing RAG.

At its core, RAG is a technique where a Large Language Model (LLM) queries an external index—which could be a search engine, a knowledge base, or a vector database—to gather additional, contextually relevant information for its response. This approach contrasts with AI models solely relying on their training data, which has a fixed knowledge cutoff date. When an AI model is asked about recent events or live data it wasn’t trained on, it operates from a point of memory without immediate reference material. This can lead to inaccuracies or "hallucinations," where the AI generates false or misleading information, such as suggesting harmful substances are beneficial.

RAG provides AI models with access to correct and up-to-date material, mitigating the risk of hallucinations. In this process, LLMs either supplement or override their internal knowledge, often referred to as "parametric memory," to deliver more reliable answers. This is also known as "grounding," where the AI’s response is anchored to specific sources, preventing it from merely generating content based on patterns learned during training. The RAG process can be broken down into three key stages: search, retrieve, and generate.

The concept of RAG predates the current AI search boom by several years. It was first documented in a 2020 research paper, well before the widespread adoption of tools like ChatGPT. Michael King, Founder of iPullRank, highlighted the significance of RAG for the future of search in an article published in 2023, anticipating the industry’s later attention to this technology. Most current AI tools utilize both RAG and their trained knowledge in conjunction.

Under the hood, AI tools typically employ two main components: a base model that generates language based on learned patterns from training data, and a retrieval layer that actively seeks out sources to attach to the generated response. While influencing the base model’s training data is challenging, gaining visibility in the retrieval results is largely an extension of Search Engine Optimization (SEO) principles.

How RAG Works:

The RAG process involves three sequential steps: search, retrieve, and generate.

Step 1: The AI Decides Whether to Search

Before any external data is retrieved, the AI first determines if its existing knowledge base is sufficient. For straightforward factual queries, such as "What is a VPN?", the core model can often provide an answer based on its training data, negating the need for external retrieval. In the case of ChatGPT, a smaller classifier model, part of a system known as "sonicberry" according to David McSweeney, analyzes the query. This classifier assigns probability scores to determine whether the query requires no search, a simple search, or a more complex, multi-step search. Other AI tools may have different internal mechanisms, but the fundamental logic remains: not every query necessitates an external search.

Step 2: The AI Executes a Search

When a query requires additional context, ChatGPT expands it into multiple related queries and submits them to an external search index, such as Bing or Google, to gather results. This expansion process is termed "query fan-out." Newer AI systems may go further, employing an "agentic" approach where they search, analyze the returned information, and then decide if further searches are needed based on what they’ve learned. These advanced systems are not limited to web searches; a query about current mortgage rates, for example, might trigger a live API call instead of a text retrieval. Michael King further explores these "agentic RAG" concepts in his detailed guide.

Once a selection of pages is gathered, on-page SEO factors like the title, meta description, and URL play a significant role in determining which page is read in full, according to research by AI expert Dan Petrovic. Petrovic’s findings indicate that sources are shortlisted for scraping based on criteria such as relevance, authority, recency, and diversity of perspective.

Evidence suggests that some AI assistants may have preferential treatment for certain domains. AI expert Jérôme Salomon has observed that ChatGPT appears to be building its own cached content index, meaning it doesn’t always rely solely on live search engine results. Furthermore, separate research by Mark Williams-Cook, David McSweeney, and Suganthan Mohanadasan indicates that ChatGPT may also incorporate content from a licensed "VIP" tier of authoritative sites and publishers, often those with existing content agreements (e.g., Reuters, The Wall Street Journal, Wikipedia). These sources are reportedly tagged as "labrador" in ChatGPT’s network traffic and are retrieved with pre-summarized, near-full-article extracts, differentiating them from content scraped and chunked from other sources.

Step 3: Content is Chunked, and the Closest Match Wins

Before the retrieved web content can be used in the AI’s response, it is broken down into smaller segments known as "chunks." This "chunking" process is akin to dividing a book into individual chapters. The system segments the page content and then identifies the piece that most effectively answers the user’s query. The system then selects the chunk that is the closest match to the original query.