Definition
llms.txt is a proposed text file at the root of your website (/llms.txt) that gives language models a compact, Markdown-formatted summary of what your site is and which pages matter most. The proposal dates from September 2024 and comes from Jeremy Howard (Answer.AI); it is meant as a counterpart to robots.txt: not 'what may you crawl' but 'this is what you should know and where to find it'.
The file starts with an H1 containing the site name, a short description as a blockquote, and then sections with lists of links, each with a brief note. An optional section called 'Optional' marks links that can be skipped when context space is limited. The proposal also suggests offering a Markdown version of each page at the same URL with .md appended, and an extended file /llms-full.txt with all content in one document.
Honesty requires saying that adoption on the engine side is limited. None of the major players — OpenAI, Google, Anthropic, Perplexity — has confirmed that their crawlers or answer engines use llms.txt when retrieving sources, and Google has publicly stated that the file does nothing for Google Search. Documentation sites and developer tools in particular have embraced it, because AI coding assistants can work with it when a user explicitly provides it.
So why add it? Because it costs little, has no downsides and forces you to decide which pages you really want a model to know. It is also a handy input for your own AI applications and for partners working with your documentation. Treat it as good housekeeping, not as a ranking factor.
What demonstrably does matter for AI visibility: crawlable, fast pages with a direct answer, entities and structured data, and not blocking AI crawlers in robots.txt. Traze generates llms.txt automatically from the sitemap and the most important pages, but spends the real effort on the pages themselves.