What Is llms.txt? An Honest Guide, With an Example

Taha Yelkenci · Founder, rankzupAI · Aug 5, 2026 · 6 min read · Technical GEO

The llms.txt proposal in one paragraph

llms.txt is a proposed standard: a Markdown file you place at the root of your site, at /llms.txt, that gives language models a curated, clutter-free map of your most important content. The idea comes from Jeremy Howard of Answer.AI, who published the proposal in September 2024. The pitch is simple enough — HTML pages are full of navigation, cookie banners, and scripts that waste an LLM's limited context window, so give the machines a clean text menu instead: what the site is, what matters most, and where to find plain-Markdown versions of key pages.

The proposal also describes a bigger sibling, /llms-full.txt, which inlines the full text of your documentation into one giant Markdown file, so an AI tool can ingest everything in a single fetch. Some sites additionally serve individual pages as Markdown by appending .md to URLs, which is part of the same convention.

What an llms.txt file looks like

It's deliberately minimal. An H1 with the site name, a blockquote with a one-line summary, optional free-form notes, then H2-grouped link lists with short descriptions. One convention with actual teeth: a section named ## Optional marks content that can be skipped when an agent is short on context.

That's the whole spec. No XML, no new syntax, nothing to validate against beyond ordinary Markdown. You could write one in fifteen minutes for most sites, and that fact does a lot of work in the "should you bother" argument later.

Who actually supports llms.txt — honestly

This is where most articles about llms.txt get slippery, so let me separate two very different kinds of "support."

Publishing the file is genuinely common now. Anthropic serves one for its documentation. So do Cursor, Zapier, Cloudflare, Perplexity's own docs, and thousands of others — Mintlify auto-generates llms.txt for every docs site on its platform, which single-handedly created a big slice of the adoption numbers you'll see quoted. There are directories tracking adopters and generator tools in every ecosystem. Among developer-tool docs sites specifically, it's close to table stakes.

Consuming the file is a different story, and here the honest answer is deflating. No major AI company has confirmed that its assistant or crawler uses llms.txt when answering questions. Google's John Mueller said outright, in mid-2025, that no AI system he was aware of used it, and compared it to the keywords meta tag — a self-declared signal engines learned to ignore because site owners can't be trusted to describe themselves. Server-log analyses that people have published tend to show the same picture: AI crawlers fetch it rarely, if at all, and there's no solid public evidence that having one changes how often you get cited. (Testing a claim like that takes repeated sampling of the kind our scoring methodology describes, not a single before-and-after check.)

So the current state is thousands of publishers and, publicly at least, no confirmed readers among the big engines. Worth knowing before anyone sells you an "llms.txt optimization service."

The case against llms.txt

Beyond non-adoption, there's a structural criticism worth taking seriously: llms.txt is self-declared. Search engines spent twenty years learning not to trust what sites say about themselves, which is exactly why the keywords meta tag died. Any signal a site owner fully controls will be gamed the moment it carries weight, and the engines know this. Some skeptics also point out that a well-structured site with clean HTML and sitemaps already gives crawlers everything llms.txt offers.

There's also a quieter risk: a stale llms.txt that points to deleted pages or describes a product you've since pivoted away from. A file nobody on the team remembers to update is a file that eventually lies. If you add one, put it in your release checklist or generate it from your CMS.

Should you add one anyway?

I think yes, for most sites — but for unglamorous reasons.

The cost is nearly zero. Fifteen minutes to an hour, no engineering risk, no downside for search since regular crawlers just ignore it. Against that near-zero cost you're buying a cheap option on a possible future: if any major engine starts honoring the file, early adopters get the benefit on day one. And there's a real present-tense benefit that has nothing to do with ChatGPT rankings — AI coding tools and agents that fetch pages directly do sometimes use llms.txt and Markdown mirrors, which matters a lot if you sell developer tools and approximately zero if you sell garden furniture.

To make it concrete: imagine a small API company whose docs run on Mintlify. They get llms.txt for free, and users pasting docs.example.com/llms-full.txt into Claude or Cursor get accurate, current answers about the API instead of hallucinated method names. That's a genuine support-load win today, no speculation required. Meanwhile a local bakery adding llms.txt should expect nothing measurable. Same file, completely different value — the difference is whether your audience points AI tools at your site.

Just don't confuse it with strategy. The things that demonstrably move AI visibility — being cited by sources engines trust, clean structured content, real reviews and community presence — are slow and hard. llms.txt is fast and easy, which is precisely why it can't be where the game is won.

rankzupAI's visibility score panel, combining measurable signals like mention rate, position and share of voice into one score. — llms.txt isn't a proven signal; the work that actually moves visibility is the kind you can measure and track in a panel like this.

A full llms.txt example

Here's what one might look like for a fictional invoicing SaaS:

# Ledgerly

> Ledgerly is invoicing software for freelancers and small agencies.
> It handles recurring invoices, multi-currency billing, and EU VAT.

Ledgerly's docs cover the web app and the REST API. The API uses
bearer-token auth; all endpoints are under https://api.ledgerly.example/v2.

## Docs

- [Quickstart](https://ledgerly.example/docs/quickstart.md): Account
  setup to first invoice in ten minutes
- [API reference](https://ledgerly.example/docs/api.md): Every endpoint,
  with request and response examples
- [VAT guide](https://ledgerly.example/docs/vat.md): How Ledgerly
  handles EU VAT rules and reverse charge

## Guides

- [Recurring invoices](https://ledgerly.example/guides/recurring.md):
  Schedules, retries, and dunning emails
- [Migrating from spreadsheets](https://ledgerly.example/guides/import.md):
  CSV import walkthrough

## Optional

- [Changelog](https://ledgerly.example/changelog.md): Release notes
- [About](https://ledgerly.example/about.md): Company background

Drop your version at your domain root, make sure it returns as plain text, and add a line to whatever checklist keeps your sitemap honest. Then, if you're curious what AI crawlers actually see when they hit your pages — which matters far more than any manifest file — run your site through the crawler simulator and look at it from their side.

This is one piece of the broader technical GEO checklist.

Frequently asked questions

Do AI engines actually use llms.txt?

Publicly, no confirmed reader among the big engines. Publishing the file is common — Anthropic, Cursor, Cloudflare and thousands of docs sites serve one — but no major AI company has confirmed its assistant uses it, and Google's John Mueller compared it to the keywords meta tag. Treat any 'llms.txt optimization service' with that in mind.

Should I add an llms.txt to my site anyway?

If you can keep it current, the cost is low — fifteen minutes for most sites. But the real risk is a stale file that points to deleted pages or a product you've pivoted away from, and a file nobody updates eventually lies. If you add one, put it in your release checklist or generate it automatically.

What's the difference between llms.txt and llms-full.txt?

llms.txt is a minimal Markdown map — site name, a one-line summary, and grouped link lists pointing to your key pages. llms-full.txt is the bigger sibling that inlines the full text of your documentation into one giant file, so an AI tool can ingest everything in a single fetch.