AI SEO. Emerging Standards
llms.txt Explained: Should Your Website Have One Yet?
Every few years the web grows a new plain text file at the domain root. robots.txt told crawlers where not to go. sitemap.xml told them where everything was. The newest proposal, llms.txt, wants to tell AI systems what your site is about and where your most important content lives, in clean markdown they can digest without fighting your page templates.
The honest status in mid 2026: it is a proposal with growing but uneven adoption, and no confirmed guarantee that the major AI providers consistently consume it. So should you bother? My answer for most clients is yes, and the reasoning has less to do with the file than with what producing it forces you to do.
What llms.txt Actually Is
A markdown file at your domain root containing a short description of your site and a curated, annotated list of your most important pages, optionally with companion markdown versions of key content. Think of it as a manually curated sitemap written for language models: not everything, just what matters most, described clearly.
The Case For and Against, Honestly
“The file takes an hour. The thinking it forces, deciding which twenty pages define your business and describing each in one clear sentence, is worth the hour even if no crawler ever reads it.”
Ram Kr Shukla, SEO and Growth Consultant
If You Do It: The Right Way in Five Steps
Implementation checklist:
- Write one paragraph describing what your business is and who it serves, in plain language with your standard entity description
- Curate 15 to 25 URLs maximum: services, key comparisons, best guides, about. Curation is the point; dumping the sitemap defeats it
- Annotate every link with one sentence on what the page covers
- Keep it synchronised with reality: assign an owner and a quarterly review
- Pair it with the fundamentals that AI systems verifiably do use today: clean HTML, schema, and server-rendered content
Priority check before you start: if your content is client-side rendered or your schema is broken, fix those first. llms.txt is a refinement on top of machine-readable foundations, not a substitute for them.
Want your site AI-legible from the foundations up?
AI readiness is part of my AI SEO engagements: rendering, schema, entity consistency, and yes, a properly curated llms.txt at the end of it.
Explore AI SEO ServicesBook a Free Strategy CallRelated reading: AI SEO for dentists. Also relevant: generative engine optimisation.
Tags: llms.txtAI SEOStandardsMachine Readability




