llms.txt: What It Is, Who Reads It, Do You Need It?

llms.txt is a plain Markdown file at the root of a website (/llms.txt) that gives language models a short, curated summary of the site and links to its most useful pages. It was proposed by Jeremy Howard of Answer.AI in September 2024. As of October 2026, no major AI search engine has said it uses llms.txt to choose what to cite, and Google says outright that you don't need one. It is still worth adding if you publish developer documentation, because coding assistants and agents do use it, and it costs an hour.

What does an llms.txt file look like?

The specification at llmstxt.org defines a small, ordered Markdown structure. Only the H1 is required:

1. 1.An H1 with the name of the project or site.
2. 2.A blockquote with a short summary containing the key information needed to understand the rest of the file.
3. 3.Optional free-form Markdown (paragraphs, lists) with more detail, but no headings.
4. 4.Optional H2 sections, each containing a list of links in the form [name](url): optional notes.
5. 5.An H2 section named “Optional,” by convention, for secondary links an agent can skip when its context window is tight.

# Acme Analytics

> Acme is a privacy-first web analytics service for SaaS teams. One script tag,
> no cookies, revenue attribution from Stripe. Hosted in the EU.

Pricing is event-based. Every feature is on every plan.

## Docs

- [Install the script](https://acme.example/docs/install.md): one tag, any framework
- [Custom events](https://acme.example/docs/events.md): track signups and conversions
- [REST API](https://acme.example/docs/api.md): read stats programmatically

## Product

- [Pricing](https://acme.example/pricing): plans and limits
- [Changelog](https://acme.example/changelog): what shipped recently

## Optional

- [Blog](https://acme.example/blog): long-form articles

The proposal also recommends serving clean Markdown versions of pages, either at the same URL with .md appended or with a link rel="alternate" type="text/markdown" pointing to them, so an agent can read a page without parsing navigation, scripts and footers. Our llms.txt generator produces a valid file from a list of your pages.

What is llms-full.txt?

llms-full.txt is a convention that grew up alongside the spec, popularized by documentation platforms: instead of links, it contains the full text of the docs concatenated into one Markdown file. An agent can load the whole documentation set in one request. It is not part of the core proposal, files can get very large, and it only makes sense for documentation, not marketing sites.

Does Google use llms.txt?

No. Google's position has been consistent and has hardened over time:

• In 2025, Google's John Mueller said that, as far as he knew, none of the AI services had said they were using llms.txt, and compared it to the keywords meta tag: a site owner's claim about what the site is about, which a system would have to verify against the real content anyway.
• In July 2025, Gary Illyes of Google Search said Google does not support llms.txt and is not planning to.
• Google's guide to optimizing for generative AI features, last updated 10 July 2026, states that you don't need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, including its generative AI features.

There is one wrinkle. Google's Chrome team shipped an experimental “agentic browsing” category in Lighthouse that includes an llms.txt check, framed as helping browser agents understand a site rather than as a search signal. So llms.txt has a place in Google's thinking about agents, just not in Search ranking or AI Overviews.

Do ChatGPT, Claude and Perplexity read llms.txt?

Not as a documented input. OpenAI's crawler documentation covers GPTBot, OAI-SearchBot and ChatGPT-User and points site owners at robots.txt; Perplexity's and Anthropic's documentation do the same for their bots. None says llms.txt influences what gets indexed or cited.

Independent evidence points the same way. Server-log analyses published through 2025 and 2026 report that the major AI crawlers request llms.txt rarely or not at all compared with robots.txt and regular pages, and correlation studies across large sets of domains, including one by SE Ranking covering about 300,000 domains, found no measurable link between having an llms.txt file and being cited by AI. Individual site owners do occasionally see GPTBot or other crawlers fetch the file, which is unsurprising: crawlers fetch many URLs that do not affect ranking.

Who actually benefits from llms.txt?

The clearest use case is developer documentation consumed by coding assistants. When a developer asks an IDE agent to integrate your API, the agent needs your docs in its context window. A curated llms.txt, plus Markdown versions of each page, lets it fetch the right pages without wading through HTML. Several documentation platforms generate llms.txt and llms-full.txt automatically for this reason, and many developer-tool companies publish them.

A second, smaller audience is users who paste a URL into an assistant. When a chatbot fetches a page on a user's behalf, a clean Markdown version can produce a better summary than a heavy HTML page. That benefit is real but hard to measure, and it does not depend on llms.txt being a ranking factor.

Why hasn't llms.txt taken off with search engines?

The objection from search engines is structural, not a matter of time. A search or answer engine has to fetch and evaluate your real pages anyway, because that is what it ranks and quotes. A separate, self-described summary adds little information the engine does not already have, and it creates an incentive to say one thing in the summary and another on the page. That is the same reason the keywords meta tag stopped mattering two decades ago: anything a site owner declares about itself without it being visible to users is easy to game.

Agents are different because their constraint is not trust but context. A coding assistant with a limited context window benefits from a compact map of where the answers are, and the user who invoked it has already decided to trust your documentation. That is why llms.txt adoption has been strongest on docs sites and developer tools, and why it is more useful to think of it as an agent-facing table of contents than as an SEO file.

Should you add an llms.txt file?

How do you add llms.txt in practice?

1. 1.Decide the audience. For a docs site, the file is a table of contents for agents. For a marketing site, it is a one-screen summary: what the product is, who it's for, pricing model, key pages.
2. 2.Write the H1 and the blockquote first. Make the summary factual and specific: product category, main capabilities, pricing model, where it is hosted. This is the paragraph a model is most likely to quote.
3. 3.List your most important pages under two or three H2 sections, each with a one-line note. Ten to thirty links is plenty; this is curation, not a sitemap.
4. 4.Put secondary material under “Optional.”
5. 5.Serve the file at /llms.txt with content-type text/plain or text/markdown, generated from the same source as your sitemap so it never goes stale.
6. 6.If you have docs, serve Markdown versions of each page and link to those URLs from the file.
7. 7.Make sure robots.txt and your CDN allow the bots you want, since llms.txt grants no access on its own. The AI crawler robots.txt generator helps.
8. 8.Track requests to /llms.txt by user agent for a month and decide whether it is worth maintaining.

VisitTrack publishes its own llms.txt, generated from the same registry as its sitemap, and its API and MCP docs are written for both humans and agents. That is the pragmatic posture: maintain the file because it is nearly free, and spend the real effort on the things that do move AI visibility, covered in GEO for SaaS.

What are the common llms.txt mistakes?

• Treating it as access control. llms.txt does not allow or block anything; robots.txt does.
• Dumping every URL. A file with 2,000 links is a worse sitemap, not a better summary.
• Marketing copy in the summary. “The world's leading platform” tells a model nothing. State what the product does and for whom.
• Letting it go stale. A file that lists last year's pricing or deleted pages is worse than none, because it contradicts your site.
• Hiding content in it. Putting information in llms.txt that is not on your real pages invites the exact verification problem Google's Mueller described.