AI Citations: Understanding Volatility and Stability in AI-Generated Content

A comprehensive study examining AI citation behavior across five major AI tools over a 20-day period reveals significant volatility in how often websites are referenced, offering crucial insights for content creators and marketers seeking to understand their digital visibility in the evolving AI landscape. The research, which tracked daily questions posed to tools like Google AI Mode, ChatGPT, AI Overviews, Gemini, and Copilot, found that a substantial portion of citations disappear by the following day, though a majority eventually return. The stability of these citations is influenced by a complex interplay of factors, including the AI tool itself, the nature of the query, and the performance of the cited web page in organic search results.

The study’s methodology involved a colleague posing identical questions to five AI tools daily for 20 days. Each website cited on day one was then monitored for its continued presence in subsequent daily responses. This data was further validated by analyzing daily tracked queries within Ahrefs’ Brand Radar tool, focusing on AI visibility, marketing conferences, and SEO tools, to confirm patterns beyond the initial 20-day observational window.

Part 1: The Transient Nature of AI Citations

A primary finding indicates that AI citations are often ephemeral. Across the surveyed AI tools, at least half of all citations vanished by the next day. Specifically, Google AI Mode and ChatGPT saw around 60% of their citations disappear, AI Overviews around 50%, and Gemini and Copilot an alarming 80%. This churn is partly attributable to the AI tools’ internal processes, which involve multiple web searches (fan-out searches) before generating an answer. Even when the search queries remained consistent from one day to the next, ChatGPT’s cited websites changed 81% of the time. This suggests that the decision-making process of which pages to cite occurs after the search results are compiled. This phenomenon was corroborated by other data, where AI Overviews cited roughly half the same pages on consecutive days, and ChatGPT approximately a quarter. Consequently, the study advises against judging AI visibility based on a single check, recommending instead a seven-day average to gauge consistent citation.

Despite this high rate of disappearance, a significant majority of lost AI citations show a tendency to return. Over the 20-day study period, approximately 75% of citations that had vanished at least once reappeared. A notable portion, around half, returned within three days, with Google AI Overviews and ChatGPT seeing about 40% of citations return after just one day of absence. Only a minority, between 20% and 27% depending on the tool, disappeared permanently during the study. This suggests that a missing citation is not necessarily a permanent loss, and pages should not be drastically altered solely due to a single day’s absence. The study cautions that citations disappearing towards the end of the observation period had less time to demonstrate their return.

Furthermore, the longevity of a citation before its initial disappearance appears to be a predictor of its likelihood to return. Citations that lasted only a single day were less likely to reappear, with about a third never returning. Conversely, citations that had remained for eight to fourteen days had a near-certain chance of returning. This observation aligns with the Lindy effect, where the longer something has survived, the greater its future survival prospects. New citations, in particular, proved unstable, with around 70% disappearing the next day and only a fraction of those returning. This highlights the importance of waiting several days before considering a new citation a lasting achievement.

Part 2: Distinguishing Real Losses from AI Quirks

The study also identified nuances that can make a perceived loss in citation not a true reflection of a page’s declining relevance. One significant factor is that certain AI tools, notably Gemini and Copilot, frequently produce answers without any source citations at all. More than half of the responses from these two tools lacked citations, which artificially inflated their citation disappearance rates. When these "empty" answers were excluded from the analysis, Gemini and Copilot’s citation stability became more comparable to that of Google’s tools and ChatGPT, with around 60% of citations disappearing by the next answer that did include sources. For Google’s tools and ChatGPT, excluding empty answers had a minimal impact as they almost always provided citations within the studied dataset. This implies that for Gemini and Copilot, it is crucial to first ascertain whether any source was cited at all before interpreting a missing citation as a loss.

Another critical distinction lies in the difference between a website citation and a brand mention. AI tools can link to a website, mention a brand by name, or both. The research found that most cited websites were not explicitly mentioned in the AI-generated text. Even when a brand was both cited and mentioned, one could disappear while the other remained. Across Google’s tools and ChatGPT, a notable percentage of pages lost their website citation but retained the brand mention, or vice versa. Gemini and Copilot showed a higher propensity for losing the website citation while keeping the brand mention, partly due to their frequent lack of overall citations. Brand names, however, tended to be more persistent than website citations, with a significantly higher percentage of brand names appearing on multiple days compared to cited websites. The study also accounted for variations in brand naming conventions, using AI to group similar brand names to avoid falsely counting these as losses. This emphasizes the need to track website citations and brand mentions separately and to verify if a different version of a brand name was used before declaring a mention lost.

Part 3: Factors Influencing Citation Stability

The research delved into page characteristics and query types that correlate with more stable AI citations. Google rankings and organic search traffic emerged as stronger indicators of citation longevity than traditional SEO metrics like Domain Rating (DR) or the number of backlinks. Pages that already ranked in Google and received organic search traffic tended to retain AI citations for longer periods. For instance, pages cited for 15 days or more on AI Overviews had significantly higher median monthly organic visits compared to those cited for only one day. While Domain Rating and backlinks showed a weaker correlation, pages ranking within Google’s top 100 for the queried topic were substantially more likely to maintain citations for at least a week, with higher rankings offering a further advantage. Pages with greater existing organic traffic also demonstrated a tendency to retain citations longer. It’s important to note that ranking is not a prerequisite for being cited, as a substantial percentage of pages cited by AI Mode and ChatGPT did not rank organically for the same queries. The findings suggest a strong association between ranking and citation stability, with ranking appearing more critical than the exact position for Google AI Overviews and AI Mode. Link building efforts should therefore be prioritized to support search rankings rather than solely aiming for AI visibility.

A closer match between a page’s title and description and the user’s query was also associated with slightly longer-lasting citations on Google’s AI Overviews and AI Mode. Pages with higher match scores (indicating a closer resemblance to the query) were more likely to be cited for extended periods. This effect was less pronounced on ChatGPT, where a clearer pattern only emerged for a small percentage of pages that remained cited for over a week. This suggests that for Google’s AI tools, crafting clear page titles and meta descriptions that accurately reflect the questions you aim to be cited for can provide a marginal advantage, complementing strong content and search visibility.

When a query specifically names a brand, that brand’s website tends to retain citations more consistently, particularly for factual inquiries rather than reviews or comparisons. For factual questions about pricing, plans, or features, AI tools frequently cited the brand’s own website. However, for reviews and comparisons, third-party sources were often preferred. While branded queries generally showed more stable citations, the study noted that even for branded queries, the brand’s own website might not be cited on all days, especially if the AI prioritizes third-party reviews. This highlights the importance of maintaining clear, accurate, and easily accessible pages detailing pricing, plans, and features. For reviews and comparisons, attention should be paid to the third-party websites that AI tools consistently cite.

Certain types of websites demonstrated exceptional citation stability across all 20 days, including YouTube, Reddit, and major publishers. This suggests that for specific citation spots, it may be more pragmatic to aim for inclusion within this existing content rather than attempting to replace it.

Finally, the study found that the type of question significantly impacts citation variability. "Best X" questions exhibited the most changeable AI citations, with approximately 75% of cited pages changing daily. This contrasts with brand-related queries, which showed more stability. This variation is likely due to the competitive landscape, where a plethora of similar pages vie for limited citation spots in "best X" scenarios, whereas health-related questions, for example, may draw from a smaller pool of trusted sources. This implies that different expectations should be set for different types of queries, acknowledging that "best X" questions are inherently more dynamic.

The Ahrefs Brand Radar tool offers a robust platform for tracking AI citation performance, providing detailed insights through its Cited pages report. Users can filter by AI platform, examine citations from their own site, competitors, and other websites, and monitor historical trends. Key metrics include citation frequency, the number of days cited, and the overall value of each citation. The watchlist feature functions akin to rank tracking for AI citations, allowing users to monitor specific pages or competitor performance across tracked prompts. Additionally, AI-powered querying within Brand Radar can provide quick answers to complex data analysis questions, such as average citation rates over a tracking period.

In conclusion, while AI citations can be a valuable indicator of visibility, their inherent volatility, particularly for non-branded and comparative queries, necessitates a nuanced approach to interpretation and strategy. Understanding the factors that contribute to citation stability, distinguishing between true losses and AI-driven variations, and leveraging robust tracking tools can empower businesses to navigate this dynamic digital landscape effectively. For branded queries, identifying citation gaps where a brand’s website was not retrieved by the AI presents a direct opportunity for improved visibility.