Glossary · SEO & AI search

What is llms.txt?

llms.txt is a proposed standard for a Markdown file served at a website's root (/llms.txt) that gives large language models and AI tools a concise, curated summary of the site and links to its most useful content.

It was proposed by Jeremy Howard of Answer.AI in September 2024, as a companion to robots.txt and sitemaps aimed at language models rather than crawlers.

Also called: llms-full.txt

Updated

llms.txt format

# VisitTrack

> Cookie-free web analytics with revenue attribution.

## Docs
- [Install](https://example.com/docs/install): One script tag.
- [Cookieless mode](https://example.com/docs/cookieless): How it works.

## Optional
- [Blog](https://example.com/blog)

The spec asks for an H1 with the site or project name, a blockquote summary, optional free-text detail, then H2 sections of Markdown link lists with short descriptions. A section named "Optional" marks links that can be skipped when context is short. Some sites also publish llms-full.txt with the full text of their docs in one file.

Does llms.txt actually work?

Adoption has been strongest among developer tools and documentation sites, and some AI coding assistants and IDE tools read these files when pointed at them. But no major search engine has committed to using it for ranking or AI answers; Google representatives have said Google Search doesn't use it. Treat it as cheap, harmless documentation for AI tools — not a ranking factor and not a substitute for crawlable HTML.

llms.txt example use

A developer asks an AI coding assistant to integrate your API and gives it your docs URL. With an llms-full.txt, the assistant can read your whole reference in one request instead of crawling dozens of pages, and is more likely to use current endpoints. That's where llms.txt earns its keep today.

llms.txt pitfalls

  • Letting it go stale — an outdated file is worse than none.
  • Listing everything; the point is curation.
  • Assuming it controls access. It doesn't — use robots.txt for that.
  • Expecting traffic from it.

llms.txt and VisitTrack

VisitTrack publishes its own /llms.txt, generated from the same registry as its sitemap so it never drifts. Create one for your site with the free llms.txt generator. To see whether AI tools actually fetch it — or your other pages — VisitTrack's AI crawler tracking shows each crawler's requests by path.

Frequently asked questions

Is llms.txt an official standard?

No. It's a community proposal from September 2024, not a standard from a standards body, and support varies by tool.

Does Google use llms.txt?

Google has said Google Search doesn't use llms.txt. Its AI features rely on its normal crawling and indexing, controlled through robots.txt and tokens like Google-Extended.

Related terms

Tools and guides

See which channels actually bring paying customers

VisitTrack is cookie-free analytics with revenue attribution built in. One script tag, no consent banner, live in two minutes. 14 days free, no card required.