{
 "acronym": "",
 "adoption": {
  "by": "Rankability (Tranco top-1000, June 2026: 8.7%) and SE Ranking (~300,000 domains analyzed: 10.13%) — independent third-party publisher-side counts, not our own probe",
  "level": "claimed",
  "probe_run": null
 },
 "aliases": [
  "llmstxt.org"
 ],
 "analog": "sitemap.xml / a README for a site's documentation",
 "collisions": [],
 "discovery": {
  "mechanism": "Fixed filename at the site root or at any subpath, covering the pages under that path",
  "path": "/llms.txt (or /<path>/llms.txt, e.g. /docs/llms.txt)"
 },
 "dispute": {
  "claims": [
   "llmstxt.org and hosting-platform vendors (Mintlify, GitBook, Wix, Yoast, AIOSEO) present growing publisher adoption as evidence of success",
   "Google Search Central states plainly that llms.txt is unnecessary for its generative AI features and lists it as a myth to ignore",
   "Ahrefs' traffic-log study found 97% of published llms.txt files are never read by any client"
  ],
  "summary": "Publisher-side adoption (thousands of sites, roughly 8-10% of measured domain samples) is separate from and should not be conflated with consumption: no answer engine has documented reading llms.txt, Google explicitly says its generative AI features don't use it, and a large-scale log analysis found the great majority of published files are never fetched."
 },
 "evidence": [
  {
   "claim": "Full v2 proposal text: file grammar, discovery link relations (rel=\"alternate\" type=\"text/markdown\", rel=\"describedby\"), relationship to robots.txt/sitemap.xml, and that AI labs (OpenAI, Anthropic, Gemini) publish llms.txt for their own docs",
   "date": "2026-08",
   "label": "VERIFIED",
   "url": "https://llmstxt.org"
  },
  {
   "claim": "v2 (August 2026) changelog: adds standard link relations for discoverability, allows both `page.html.md` and `page.md` forms, defines what a subpath file covers, and drops the v1 context-expansion tooling and the mechanical meaning of the \"Optional\" section",
   "date": "2026-08",
   "label": "VERIFIED",
   "url": "https://llmstxt.org/changes.html"
  },
  {
   "claim": "Google's official Search Central guidance (last updated 2026-07-10) lists \"LLMS.txt files and other 'special' markup\" under things site owners can ignore, stating Google Search's generative AI features do not use them",
   "date": "2026-07-10",
   "label": "VERIFIED",
   "url": "https://developers.google.com/search/docs/fundamentals/ai-optimization-guide"
  },
  {
   "claim": "8.7% of the Tranco top-1,000 domains and 5.6% of the top-10,000 published a valid llms.txt file as of June 2026",
   "date": "2026-06",
   "label": "REPORTED",
   "url": "https://www.rankability.com/data/llms-txt-adoption/"
  },
  {
   "claim": "Across a dataset of nearly 300,000 domains, 10.13% carried an llms.txt file, with adoption not concentrated among high-traffic sites (8.27% for 100,001+ visit sites vs 10.54% for mid-traffic sites)",
   "date": "2026",
   "label": "REPORTED",
   "url": "https://seranking.com/blog/llms-txt/"
  },
  {
   "claim": "Analysis of 137,000 sites publishing llms.txt found 97% of the files were never fetched by any client in the observation window; of requests that did reach a file, 96% came from bots rather than identifiable AI assistants",
   "date": "2026",
   "label": "REPORTED",
   "url": "https://ahrefs.com/blog/llmstxt-study/"
  }
 ],
 "form": "well-known-file",
 "governance": "Jeremy Howard / Answer.AI; maintained as an open proposal in the AnswerDotAI/llms-txt GitHub repository with a public Discord for implementation discussion",
 "group": null,
 "history": [
  {
   "date": "2026-09-05",
   "note": "seeded from Exa Agent research run"
  },
  {
   "date": "2026-09-06",
   "note": "verified against llmstxt.org (v2, Aug 2026), llmstxt.org/changes.html, Google's AI optimization guide, and three independent third-party adoption studies (Rankability, SE Ranking, Ahrefs)"
  }
 ],
 "id": "llms-txt",
 "implementation": "multi-implementation",
 "last_verified": "2026-09-06",
 "name": "llms.txt",
 "native_status": "\"This is v2 of the proposal\" — explicitly a community proposal open for input, not a standards-track document",
 "origin": {
  "date": "2024-09",
  "org": "Jeremy Howard / Answer.AI"
 },
 "part_of": null,
 "parties": [
  "agent-site"
 ],
 "phase": "draft",
 "probe": {
  "method": "GET",
  "on_absent": "No llms.txt at this path; a more specific subpath file (e.g. /docs/llms.txt) may still exist and take precedence for that subpath",
  "parse": "200 response beginning with an H1 heading (`# <project/site name>`) followed by a blockquote summary, per the llms.txt v2 grammar",
  "path": "{origin}/llms.txt"
 },
 "question": "Where can an agent find a concise, curated map of a site's content, written for language models rather than browsers?",
 "scope": "general",
 "spec_url": "https://llmstxt.org",
 "step": "read",
 "summary_md": "llms.txt is a proposed convention for publishing a small, curated, markdown-formatted index of a site's content at `/llms.txt` (or a subpath such as `/docs/llms.txt`), intended to be read by language models and AI agents rather than crawled and scraped like a full site [1]. The file is one required H1 heading naming the project, an optional blockquote summary, optional free-form context, and then H2-delimited \"file lists\" of markdown links to more detailed content — typically clean `.md` mirrors of a site's HTML pages [1].\n\nIt solves a narrow problem: HTML pages are built for browsers, wrap information in navigation and scripts that are wasteful to parse into an LLM's limited context window, and give an agent no small, authoritative starting point for a large site. llms.txt gives a site a place to say \"start here\" and to link to markdown-native content instead of full HTML [1].\n\nIt does not solve discovery for sites that don't publish one, does not replace `robots.txt` (a permission signal, not a content index) or `sitemap.xml` (exhaustive, for search-engine crawlers, and usually not markdown-native) [1], and — critically — it does not guarantee that anything reads it. Google's own Search Central documentation, last updated 2026-07-10, explicitly tells site owners they \"don't need to create ... llms.txt\" because \"Google Search itself doesn't use them,\" filing it alongside other AEO/GEO \"hacks\" to ignore [3]. No answer engine has published documentation describing llms.txt consumption in its retrieval pipeline.\n\nThe current text is v2, published August 2026, revising the original September 2024 proposal by Jeremy Howard (Answer.AI). v2's main addition is standard discovery link relations (`rel=\"alternate\" type=\"text/markdown\"` for a page's markdown twin, `rel=\"describedby\"` for the covering llms.txt file), expressible as HTML `<link>` tags or an HTTP `Link:` header; it also loosened the markdown-mirror URL convention and dropped the v1 context-expansion tooling and the special meaning of the \"Optional\" section [1][2]. The specification remains an informal, single-maintainer GitHub proposal, not a standards-track document of any kind [1].\n\nOn the producer side, several documentation platforms and CMSs generate llms.txt automatically — Mintlify, GitBook, Yoast SEO, AIOSEO, and Wix — and OpenAI, Anthropic, and Google publish llms.txt for their own developer docs [1]. This is genuine, if modest, publisher-side adoption: independent Tranco-based studies put it at roughly 8.7% of the top 1,000 domains (June 2026) and 5.6% of the top 10,000 [4], while a broader ~300,000-domain analysis found 10.13% overall, spread fairly evenly across traffic tiers rather than concentrated among high-traffic sites [5].\n\nWhat is disputed or unknown is consumption, not existence. The gap between \"a file exists\" and \"a model reads it\" is large and, per the sources available, mostly unmeasured by anyone with visibility into model retrieval internals. One large-scale server-log study of 137,000 llms.txt-publishing sites found 97% of the files were never fetched by any client during the observation window, and of the requests that did arrive, 96% came from generic bots rather than identifiable AI assistants [6]. Combined with Google's explicit statement that its own systems ignore the file, the safest characterization is: llms.txt is a real, growing publisher-side convention with no documented reader among major answer engines as of mid-2026.",
 "track": "single-vendor",
 "version": {
  "date": "2026-08",
  "label": "v2"
 }
}