llms.txt Explained: What It Is, and Whether It Actually Does Anything (2026)

**robots.txt grants or denies access.** It is an old, universally honoured convention that tells crawlers which paths they may fetch. It is permission.

**llms.txt offers curation.** It says: here are the pages that best explain this site, in a clean markdown list, so a model does not have to crawl and strip a hundred templated pages to understand you. It is a recommendation, not a rule, and it grants nothing that robots.txt has not already allowed.

No major answer engine has publicly committed to reading llms.txt as a ranking or retrieval input. Anyone telling you it is a ranking factor is ahead of the evidence.

It is still cheap and reasonable to publish. The file takes minutes, costs nothing to maintain if you generate it, and the exercise of choosing your twenty most explanatory pages is itself clarifying. Publish it as a low-cost bet on a convention that may matter, not as an optimisation you expect to move numbers.

Start with an H1 that is your business name and a one-line blockquote description — the same sentence you use everywhere else, because entity consistency is the actual payoff. Then group links under plain headings: what the product does, your free tools, your guides, your documentation.

Curate ruthlessly. A file listing every URL is just a sitemap in worse clothing. Twenty pages that genuinely explain you beat two hundred that do not, and each link deserves a short description of what a reader would learn from it.

Keep it generated from the same source as your sitemap if you can, so it never rots. A stale llms.txt pointing at dead pages is worse than none.