Google Official Dismisses LLMs.txt as Redundant, Similar to Meta Keywords Tag

Google’s John Mueller has expressed skepticism about the proposed LLMs.txt standard, likening its functionality to the now-defunct keywords meta tag.

This perspective aligns with feedback from webmasters who have attempted to implement LLMs.txt without observing significant benefits.

Understanding LLMs.txt

LLMs.txt has been introduced as a tool to present website content specifically to large language models and AI crawlers, but its practicality is being questioned by industry experts.

Definition and Purpose of LLMs.txt

At its core, LLMs.txt aims to streamline content delivery to AI systems by removing extraneous elements.

Unlike robots.txt, which directs how bots interact with a site, LLMs.txt focuses on providing clean, main content in a markdown format. This format uses symbols like the pound sign (#) for headings and the minus sign (-) for lists, ensuring both human and machine readability.

The idea is to present content devoid of advertisements and navigation elements, facilitating easier consumption by language models.

Despite its intended purpose, the adoption of LLMs.txt has been minimal, with only specialized bots recognizing and utilizing the file appropriately.

Google’s Take on LLMs.txt

During a recent discussion, John Mueller addressed the effectiveness of LLMs.txt, drawing parallels to previous meta tag practices.

Comparison to the Keywords Meta Tag

Mueller highlighted the similarities between LLMs.txt and the once-popular keywords meta tag, emphasizing the limited impact of both.

He noted that major AI services like Anthropic, OpenAI, and Google itself have not adopted LLMs.txt, rendering it akin to the obsolete keywords meta tag. The latter was never widely utilized by search engines due to reliability concerns, and Mueller suggests that LLMs.txt suffers from the same fate.

Webmasters relying on LLMs.txt may find it ineffective, as AI agents prefer directly parsing site content for context over external declarations.

Redundancy and Potential for Misuse

Beyond its limited adoption, LLMs.txt may introduce unnecessary complications in content management.

Mueller pointed out that since the main content and structured data are already accessible, introducing an LLMs.txt file becomes redundant.

Additionally, there’s a risk of malicious actors using LLMs.txt to present misleading information to AI agents, creating discrepancies between what users see and what AI interprets. This potential for cloaking undermines trust and effectiveness, further questioning the utility of LLMs.txt.

Consequently, without support from leading AI platforms, LLMs.txt remains an unadopted and questionable standard in the SEO community.

The Bottom Line

John Mueller’s assessment underscores the limited value of LLMs.txt, likening it to the ineffective keywords meta tag.

Without backing from major AI services and with potential drawbacks in usage, webmasters might find it more advantageous to focus on optimizing their site content directly rather than relying on this proposed standard.