# llms.txt

llms.txt is a proposed standard (analogous to robots.txt) for a markdown file at a website's root that gives large language models a structured summary of the site's content and key pages. The proposal was introduced by Jeremy Howard in September 2024 at llmstxt.org.

Canonical: https://myleadsfactory.com/glossary/llms-txt

## In depth

- Format: H1 with site name, blockquote summary, H2 sections listing key pages with descriptions, followed by optional H2 'Optional' section with deeper material.
- Adoption is still mostly opt-in: Anthropic's tooling reads it, but ChatGPT crawlers and Google don't yet (as of mid-2026).
- Companion file llms-full.txt expands the same structure with full body text, useful when an LLM needs the actual content, not just navigation.
- Practical value today: modest. Cost to deploy: trivial (a single text file). Recommended as a low-effort optimization even though impact is currently limited.

## Common misconception

llms.txt is NOT a way to prevent LLMs from crawling your site, that's robots.txt with bot-specific disallow rules (GPTBot, ClaudeBot, PerplexityBot, etc.). llms.txt is the opposite: it INVITES LLMs to consume a structured summary.

Source: [llmstxt.org: The /llms.txt proposal](https://llmstxt.org/)

## Related terms

- https://myleadsfactory.com/glossary/aeo
- https://myleadsfactory.com/glossary/robots-txt