Does adding an llms.txt file improve AI search visibility?

No: the current evidence shows no measurable positive effect on AI search visibility.

An OtterlyAI study tracked 62,100+ AI bot visits over 90 days across sites with and without llms.txt files. The finding: no positive correlation between llms.txt presence and increased AI crawler activity or citation rates.

An SE Ranking XGBoost model trained to predict AI citation rates on approximately 300,000 domains found removing llms.txt as a variable actually improved prediction accuracy: suggesting the file provides no signal that helps predict whether a site gets cited. If llms.txt were working as intended, removing it from a predictive model would reduce accuracy, not improve it.

Why llms.txt doesn't work the way robots.txt does

robots.txt works because it has consequences: AI crawlers that ignore robots.txt risk being blocked at the network level, so there is compliance incentive. llms.txt is advisory. It tells AI systems what content to prioritise: but provides no mechanism to enforce that priority, and no major AI platform has published documentation confirming they parse or act on llms.txt content.

Answer.AI proposed the format in 2024. As of 2026, neither OpenAI, Anthropic, Google, nor Perplexity has confirmed llms.txt implementation. A file with no confirmed readers produces no measurable effect.

The confusion with robots.txt

A Writesonic analysis of 6 AI crawlers found title tags (readable by 5/6 crawlers) and meta descriptions (readable by 4/6) are the actual high-readability content signals. JSON-LD structured data scored 0/6 for readability across all crawlers tested: and llms.txt, which is advisory plain text rather than structured HTML, is even further from how AI crawlers actually process pages.

AI crawlers read HTML. They follow HTTP responses. They respect robots.txt because robots.txt affects crawl access. They do not follow advisory text files with no protocol enforcement.

What to do instead

The fixes that actually improve AI crawler access: allowlist AI crawlers in robots.txt. PerplexityBot, GPTBot, OAI-SearchBot, ClaudeBot, Google-Extended, and Bingbot must each be explicitly allowlisted: each is controlled independently. Google officially added the Google-Agent crawler in 2026 for its AI systems; this must be allowlisted separately from GoogleBot.

73% of sites have at least one technical barrier blocking AI crawler access (OtterlyAI): robots.txt misconfiguration is the largest single category. Second, serve content as static HTML. Five of seven major AI crawlers cannot render JavaScript. If your content is generated client-side, AI systems may not be reading it regardless of what your llms.txt says.

What the evidence doesn't settle

The OtterlyAI and SE Ranking studies are negative results: they show llms.txt has no measurable effect based on current data. If major AI platforms implement llms.txt support in the future, the finding would need to be revisited.

The format may have value for internal documentation or communicating site structure to developers building AI integrations. That use case is separate from search citation visibility. For search citation purposes, the current evidence is clear: llms.txt is not a measurable AI search signal.