What is llms.txt (with a real example)

Short answer

llms.txt is a plain-markdown file at your site root that gives AI systems a curated summary of what your site is and where its most important content lives — like a table of contents written for language models.

llms.txt is a markdown file at /llms.txt, not a robots directive — it invites, it doesn’t restrict.
Adoption is real but thin: roughly one in ten sites has one, and most implementations are copy-pasted junk.
Crawl logs show light usage: major AI bots overwhelmingly still parse your HTML directly, not this file.
No engine has confirmed it affects rankings or citations — treat every claim you read about “boosting AI visibility” with llms.txt as unproven.
It still belongs on every project I run — ten minutes of effort, zero downside, and the exercise of writing it sharpens your actual content.
The real lever is the content the file points to, not the file itself.

The spec, in one minute

The format, proposed by Jeremy Howard in 2024, is deliberately simple: one markdown file at /llms.txt with an H1 (who you are), a blockquote summary, and H2 sections of annotated links. No XML, no registry, no tooling required.

A few things the spec is often misquoted on. There’s no character limit, no required schema validator, and no official body that certifies compliance — it’s a convention, not a standard in the ISO sense. The optional companion file, llms-full.txt, is meant to hold the same structure but with the actual page content inlined rather than just links, for models that can’t or won’t follow outbound URLs during inference. Most sites don’t need the full version; I only build one for clients whose core value proposition lives in content that’s hard to crawl cleanly — heavy JS rendering, gated PDFs, that kind of thing.

The other detail people miss: link annotations (the short text after each URL) aren’t decorative. If an engine does parse the file, that annotation is the only context it gets before deciding whether to follow the link. Write it like an editor writing a one-line pitch, not like alt text.

A real example

This site’s own file, trimmed:

# iNevidimka — Dima Mochalov
> SEO & AI Search Strategist. GEO, AEO, local SEO and link building
> for brands that want to be found in Google and in AI answers.

## Services
- [AI SEO](https://inevidimka.com/ai-seo/): flagship — visibility in engines that answer
- [GEO](https://inevidimka.com/generative-engine-optimization/): citations in ChatGPT, Perplexity, AI Overviews

## Proof
- [Case studies](https://inevidimka.com/case-studies/): documented growth, real numbers

Notice what’s absent as much as what’s present. There’s no “About us” fluff, no privacy policy link, no navigation dump. The file mirrors the handful of pages I’d want a human decision-maker to read if they only had ninety seconds — that’s the actual test I use when drafting one for a client: would this survive a ninety-second skim by someone deciding whether to trust the brand? If a section wouldn’t, it doesn’t belong in llms.txt even if it belongs on the site.

Now the honest part: does anyone read it?

Here is what the data says in 2026, and it’s less romantic than the hype: an SE Ranking study of 300,000 domains put adoption at roughly 10% of sites, and server-log research covering hundreds of millions of AI-bot visits found only a few hundred requests for /llms.txt — GPTBot, ClaudeBot and PerplexityBot overwhelmingly crawl your HTML directly. No major AI company has publicly committed to acting on the file.

Worth sitting with that gap for a second. Ten percent adoption against a few hundred fetches across hundreds of millions of visits isn’t “underused feature” territory — it’s “publishers built it, crawlers largely ignore it” territory. I’ve pulled server logs for clients who added llms.txt specifically to test this, and the pattern holds: the file sits there, robots.txt gets hit constantly, sitemap.xml gets hit constantly, llms.txt gets hit rarely if at all in a given month.

So why do I still ship it on every project?

It costs nothing. Ten minutes, one static file, zero risk.
It’s a hedge. If any engine starts honoring it, early adopters win by default.
It forces clarity. Writing a one-page summary of your site is a positioning exercise most brands have never done — and that clarity leaks into pages AI systems do read.

There’s a fourth reason I don’t put in client decks but will say here: llms.txt is a fast way to audit whether a site even has a coherent story. Half the time I sit down to write one, the client’s own team can’t agree on which three pages best represent the business. That disagreement is the actual problem — the missing file was just the symptom that surfaced it.

How do I build my own llms.txt?

Steps, in the order I actually do them:

List the five to ten URLs that best represent what the business does, sells, or proves — not your sitemap’s top-traffic pages, your best pages.
Write the H1 and blockquote first. If you can’t summarise the site in two sentences, fix that before touching the file.
Group links under two to four H2s (Services, Proof, Guides — whatever taxonomy matches how a customer thinks, not your internal org chart).
Annotate every link in under fifteen words. No marketing copy, no adjectives that aren’t verifiable.
Upload to the root — yourdomain.com/llms.txt, publicly accessible, no auth wall, correct text/plain or text/markdown content type.
Re-check it quarterly. Stale links inside llms.txt are worse than no file, because they’re a curated list of dead ends.

That’s the entire process. There’s no submission step, no ping endpoint, nothing to “index” — you publish it and move on.

llms.txt vs robots.txt vs sitemap.xml

These three files get confused constantly because they all live at the root and all sound like “instructions for bots.” They do almost opposite jobs.

File	Purpose	Who reliably reads it	Confirmed ranking/citation effect
`robots.txt`	Restricts or permits crawler access	Virtually all major crawlers, search and AI	Indirect — controls what can be indexed at all
`sitemap.xml`	Lists all indexable URLs for discovery	Search engine crawlers, reliably	Indirect — aids discovery and crawl efficiency
`llms.txt`	Curates a summary and top links for AI systems	Inconsistent; low confirmed fetch volume	None confirmed by any major AI provider

The practical reading: robots.txt and sitemap.xml are infrastructure you should never skip. llms.txt is closer to a business card left on a desk that may or may not get picked up. Don’t let a client’s excitement about the newest file distract from making sure the first two are actually correct — I still find broken sitemaps on sites that proudly launched an llms.txt file the same week.

What should — and shouldn’t — go into it

What belongs: your core value proposition in plain language, links to definitional or comparison content, case studies or proof pages, pricing pages if pricing is a genuine differentiator, and anything that answers “what is this company and why should I trust it” in one pass.

What doesn’t belong: your full navigation, blog category pages, legal boilerplate, anything gated behind a form, and — this trips people up — anything you wouldn’t want quoted verbatim out of context. If a model does ingest the file, it may lift phrasing directly. Write every line as if it could appear, unedited, inside someone’s answer.

What actually moves AI visibility

The unglamorous stuff: quotable definitional content, clean structure, entity consistency, and authority. That’s GEO — llms.txt is just the cherry.

Concretely, the work that correlates with actually getting cited across ChatGPT, Perplexity and AI Overviews looks like this: a direct-answer paragraph at the top of the page (the same “ansbox” pattern this article uses), consistent naming of your brand and entities across every page and every third-party mention, structured data that matches what’s visible on-page, and genuine external references — the kind of citations that make a model’s retrieval layer trust the source. None of that requires a special file. It requires the content itself to be quotable in isolation, because that’s the unit AI systems actually extract and reuse.

Common mistakes that make llms.txt useless

I see the same handful of errors repeatedly when auditing sites that already have a file in place. Listing every blog post instead of curating five to ten. Copying homepage marketing copy verbatim instead of writing plain, factual summaries. Forgetting to update it after a site migration, so half the links 404. Treating it as a place to stuff keywords, which defeats the entire point of a file meant to read as clean, human-usable summary text. And publishing it once, then never touching it again — a stale llms.txt signals neglect just as clearly as a stale sitemap does.

Where does llms.txt live?

At the site root: yourdomain.com/llms.txt. Optionally a fuller llms-full.txt alongside it.

Is llms.txt the same as robots.txt?

No. robots.txt controls crawler access; llms.txt offers curated content. One restricts, the other invites.

Will llms.txt improve my rankings?

There's no evidence it affects Google rankings, and AI crawlers rarely fetch it today. Treat it as a cheap hedge, not a tactic.

Should I still add it?

Yes — ten minutes of work for a free option on the future. Just don't expect miracles from it.

Do I need an llms-full.txt as well?

Only if your key content is hard for a crawler to parse cleanly — heavy client-side rendering, gated documents. Most sites can skip it and rely on the standard llms.txt links.

How often should I update llms.txt?

Review it quarterly or whenever you publish a new flagship page. Dead links inside a curated file are worse for trust than having no file at all.

Sources: aeo.press — The State of llms.txt in 2026, linkbuildinghq.com — Should websites implement llms.txt