llms.txt: What It Is and Whether It Matters
llms.txt is a plain-text file at the root of a site that describes what the site is, written for language models rather than for crawlers. No search or AI engine currently documents it as a requirement or a ranking factor. It costs one file to add, so it is reasonable insurance, but anyone selling it as the key to AI visibility is overstating a convention that is still young.
That is the honest summary. The detail is worth having, because the question comes up constantly and the answers online tend to be either dismissive or breathless.
What the file contains
A short markdown document, served as plain text at /llms.txt. Typically:
- The name of the business and a one-line description
- What the product or service actually does
- What distinguishes it, stated plainly
- Who it is for
- Links to the pages that matter most
- Contact details
Some sites also publish llms-full.txt with expanded content for models that want more than the summary.
The argument for adding it
When an assistant describes your business, it assembles that description from whatever it has read. Often that means third-party pages: a directory listing, a review site, a competitor's comparison page. Those sources are frequently out of date and occasionally wrong.
A first-party file gives any model that fetches it an accurate description from the source. That is worth something even if no engine treats it as a signal, in the same way that an accurate About page is worth having.
It also costs almost nothing. One route, generated from information you already have.
The argument for not overvaluing it
No major engine has published documentation saying it reads llms.txt, let alone that it affects anything. The convention was proposed by the community and adoption among model providers is unclear.
Which means claims about its impact are, at present, unsupported. If a tool or agency presents it as central to AI visibility, that is a reason for scepticism about the rest of what they are telling you.
What matters more
Three things, all of which have documented effects:
Letting the AI crawlers in. Check your robots.txt for GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended. Many sites block these, sometimes through a plugin default nobody chose deliberately. If they cannot read your pages, a summary file describing those pages will not help.
Structured data. Organization schema with real sameAs links, Article and FAQPage where they genuinely apply. These are read, documented, and affect how your content is understood.
Pages that answer questions directly. An engine quoting your site needs something quotable. A page that opens with a direct answer gives it one. A page that opens with throat-clearing does not.
Do those three before worrying about llms.txt.
If you add it
Keep it accurate above all. The entire value is being the authoritative source on what your business is. A file that overstates your coverage or describes features you do not have is worse than nothing, because it gets compared against your actual pages.
Keep it current, too. A static file describing last year's product is a liability that nobody remembers to check.
Generating it from data you already maintain, rather than writing it by hand once, is the difference between a file that stays true and a file that quietly rots.
Common questions
Is llms.txt the same as robots.txt?
No. robots.txt controls access and is universally honoured. llms.txt describes content and is honoured by nobody in particular, so far. The first one is load-bearing, the second is optional.
Will it stop AI models training on my content?
No. It is descriptive, not a permission system. Access control belongs in robots.txt, and even there, compliance varies by crawler.
Should I block AI crawlers instead? It depends on your business. A publisher protecting licensed content has a real argument for blocking. A business that wants to be recommended by assistants is blocking its own distribution.