agenticweb.wiki

agents.txt

Last checked · 31 primary sources

This page is about the agents.txt Standard published at agents-txt.com: a plain-text /agents.txt file, with an /agents.json companion, that announces which agent protocols a site supports. "agents.txt is the announcement layer. It tells an agent which protocols a site speaks." [1] Several unrelated proposals use the same file name and are not covered here: (a) Dennj Osele's earlier agents.txt for discovering AI agents (GitHub repository created 2025-02-15 [7]): "This project encourages businesses to install an agents.txt file to enable search engines to discover their AI agents and agent services." [8]; its site agentstxt.dev read "agents.txt v0.0.2 alpha We are building an open standard to create a decentralized, democratic Internet of Agents." in a Wayback capture of 2026-02-14 [9], and the domain now redirects to agents-txt.com [10], though neither project states a link between them; (b) Srijal Dutta's Internet-Draft "AGENTS.TXT: Strict Policy File for Automated Clients" (draft-srijal-agents-policy-00, 2025-10-07), a hash-verified allow/deny policy file, which has lapsed: "Document Type Expired Internet-Draft (individual)" [11][12]; (c) Kayla Cardillo's draft-car-agents-txt-wellknown-00 (2026-06-12; GitHub repository kaylacar/agents-txt created 2026-02-22), which "This document requests registration of two Well-Known URIs under the "/.well-known/" path: "agents.txt" and "agents.json"." and places the file under /.well-known/: "The "agents.txt" file MUST be served at: https://example.com/.well-known/agents.txt" [13][14][15]; IANA's Well-Known URIs registry lists neither name [16]; (d) Jasper van Veen's agents.txt of March 2026, since renamed: "This proposal has been renamed from agents.txt to agent-manifest.txt." [18]; its repository is now jaspervanveen/agents-brief.txt [19], and DreamHost put a default file with its directives on hosted sites in May 2026 [29][30]; (e) Pedro Luis Cuevas Villarrubia's AsturWebs agents.txt, "# Version: 2.0 # Author: Pedro Luis Cuevas Villarrubia" [20][21]; and (f) the access-policy file in the arXiv paper Towards an Agent-First Web (Bandara et al., 2026-06-17): "We propose agents.txt, a structured declaration file served at a well-known URL (/.well-known/agents.txt)" [22][23]. It is not AGENTS.md, the instruction file for coding agents kept in a code repository; it is not llms.txt, the Markdown briefing for language models, which the specification places one layer below it [2]; it is not robots.txt (see AI crawler tokens in robots.txt), the crawl-permission file; it is not ai.txt, an AI usage-permission file, and the specification keeps such usage preferences apart, saying of Content-Signal: "This is distinct from agents.txt and concerns training/indexing preferences, not agent interaction capabilities." [1]; and it is not an A2A AgentCard, which an agents.txt file can point to [1][2].

Overview

The agents.txt Standard proposes a short text file at the root of a website that tells visiting AI agents which agent protocols the site supports and where to find them: "agents.txt (with companion agents.json) is a lightweight, protocol-agnostic capability declaration format that publicly announces what agent-interaction protocols and features a website supports." [1]. Its directives point to payment protocols such as x402 and MPP, authorisation methods, MCP servers, skill files, A2A AgentCards, UCP profiles and WebMCP pages, while the companion agents.json carries the structured detail [1][2]. The file itself only announces; payments, sign-in and tool calls still happen in each protocol's own layer [1]. Its authors present it as the layer above robots.txt, sitemap.xml and llms.txt: "It is Layer 4 of the agent-readiness stack: discovery on top of llms.txt (Layer 3), sitemap.xml (Layer 2), and robots.txt (Layer 1)." [2]

Who uses it

Found by fetching /agents.txt live on 2026-10-29 from 60 candidate hostnames (wiki leads, search results and large sites; a sample, not a census): 5 served a file in this format, agents-txt.com itself [3] and the four below; 10 served /agents.txt files in other, unrelated formats; the rest served none. Each entry below opens with the line "# Standard: https://agents-txt.com".

Who reads it

No dated statement was found from an AI company or agent operator saying that its agents read /agents.txt in this format. The tools that check the file come from the proposal itself:

Recent coverage

See also

References

  1. agents.txt Standard, agents.txt Standard v1.0 (Status: Draft; dated "Published 2025-10-13" on the page), checked 2026-10-29: "Status Draft This document is a work in progress and subject to change." "Authors hsolosoftworks Maintainer hsolosoftworks.com Contact contact@agents-txt.com Published 2025-10-13" "agents.txt (with companion agents.json) is a lightweight, protocol-agnostic capability declaration format that publicly announces what agent-interaction protocols and features a website supports." "agents.txt is the announcement layer. It tells an agent which protocols a site speaks." "agents.txt MUST be served at <origin>/agents.txt. agents.json SHOULD be served at <origin>/agents.json." "This is distinct from agents.txt and concerns training/indexing preferences, not agent interaction capabilities." — https://agents-txt.com/spec/
  2. agents.txt Standard, About, checked 2026-10-29: "The agents.txt working group on GitHub." "Structural changes (new directives, schema fields, conformance shifts) require an RFC-style description in the pull request and two reviewer approvals." "It is Layer 4 of the agent-readiness stack: discovery on top of llms.txt (Layer 3), sitemap.xml (Layer 2), and robots.txt (Layer 1)." "Validate the output with the live audit tool or the audit_site MCP tool at mcp.agents-txt.com." — https://agents-txt.com/about/
  3. agents.txt Standard, reference file /agents.txt on agents-txt.com, checked 2026-10-29: "# Standard: https://agents-txt.com" — https://agents-txt.com/agents.txt
  4. GitHub API, agents-txt/agents-txt repository record (created 2026-05-13 UTC), checked 2026-10-29: "2026-05-13" — https://api.github.com/repos/agents-txt/agents-txt
  5. GitHub API, agents-txt/agents-txt commits (dates in UTC), checked 2026-10-29: "2026-05-10" — https://api.github.com/repos/agents-txt/agents-txt/commits?per_page=100
  6. Internet Archive Wayback Machine, capture index for agents-txt.com (first captures; used only to date the site), checked 2026-10-29: "20260518" — https://web.archive.org/cdx/search/cdx?url=agents-txt.com&output=json&limit=5
  7. GitHub API, dennj/agents.txt repository record (created 2025-02-15 UTC; homepage agentstxt.dev), checked 2026-10-29: "2025-02-15" — https://api.github.com/repos/dennj/agents.txt
  8. dennj/agents.txt, README.md, checked 2026-10-29: "This project encourages businesses to install an agents.txt file to enable search engines to discover their AI agents and agent services." — https://raw.githubusercontent.com/dennj/agents.txt/main/README.md
  9. Internet Archive Wayback Machine, agentstxt.dev as captured 2026-02-14 (the live domain now redirects, so the archived copy is used), checked 2026-10-29: "agents.txt v0.0.2 alpha We are building an open standard to create a decentralized, democratic Internet of Agents." — https://web.archive.org/web/20260214213231id_/https://agentstxt.dev/
  10. Internet Archive Wayback Machine, capture index for agentstxt.dev, checked 2026-10-29: "agentstxt.dev" — https://web.archive.org/cdx/search/cdx?url=agentstxt.dev&output=json&limit=8
  11. S. Dutta, AGENTS.TXT: Strict Policy File for Automated Clients, draft-srijal-agents-policy-00, 7 October 2025, checked 2026-10-29: "AGENTS.TXT: Strict Policy File for Automated Clients" "Malformed files are treated as fully restrictive to prevent unintended access." "The canonical path for the file is /agents.txt." — https://www.ietf.org/archive/id/draft-srijal-agents-policy-00.txt
  12. IETF Datatracker, draft-srijal-agents-policy, checked 2026-10-29: "Document Type Expired Internet-Draft (individual)" — https://datatracker.ietf.org/doc/draft-srijal-agents-policy/
  13. K. Cardillo, AGENTS.TXT: Capability Declarations for Web Agents, draft-car-agents-txt-wellknown-00, 12 June 2026, checked 2026-10-29: "This document requests registration of two Well-Known URIs under the "/.well-known/" path: "agents.txt" and "agents.json"." "The "agents.txt" file MUST be served at: https://example.com/.well-known/agents.txt" — https://www.ietf.org/archive/id/draft-car-agents-txt-wellknown-00.txt
  14. IETF Datatracker, draft-car-agents-txt-wellknown, checked 2026-10-29: "Document Type Active Internet-Draft (individual) Author Kayla Cardillo" — https://datatracker.ietf.org/doc/draft-car-agents-txt-wellknown/
  15. GitHub API, kaylacar/agents-txt repository record (created 2026-02-22 UTC), checked 2026-10-29: "2026-02-22" — https://api.github.com/repos/kaylacar/agents-txt
  16. IANA, Well-Known URIs registry (CSV), checked 2026-10-29: "URI Suffix,Reference,Status,Change Controller" — https://www.iana.org/assignments/well-known-uris/well-known-uris-1.csv
  17. IETF Datatracker API, Internet-Drafts whose name contains "agents", checked 2026-10-29: "draft-car-agents-txt-wellknown" — https://datatracker.ietf.org/api/v1/doc/document/?name__contains=agents&type=draft&format=json&limit=200
  18. Jasper van Veen, DEV Community, agent-manifest.txt — a proposed web standard for AI agents (formerly agents.txt), posted 2026-03-13, edited 2026-03-27, checked 2026-10-29: "This proposal has been renamed from agents.txt to agent-manifest.txt." "the agents.txt namespace became crowded: an independent IETF Internet-Draft (draft-srijal-agents-policy-00) had already claimed that filename, and multiple community projects were independently using it for different purposes." — https://dev.to/jaspervanveen/agentstxt-a-proposed-web-standard-for-ai-agents-20lb
  19. GitHub API, jaspervanveen/agents-brief.txt repository record (created 2026-03-09 UTC), checked 2026-10-29: "2026-03-09" — https://api.github.com/repos/jaspervanveen/agents-brief.txt
  20. AsturWebs, /agents.txt on asturwebs.es, checked 2026-10-29: "# Version: 2.0 # Author: Pedro Luis Cuevas Villarrubia" — https://asturwebs.es/agents.txt
  21. GitHub API, asturwebs/agents-txt repository record (created 2026-05-12 UTC), checked 2026-10-29: "2026-05-12" — https://api.github.com/repos/asturwebs/agents-txt
  22. Eranga Bandara and 20 other authors, Towards an Agent-First Web: Redesigning the Web for AI Agents, arXiv:2606.19116, submitted 2026-06-17, checked 2026-10-29: "Towards an Agent-First Web: Redesigning the Web for AI Agents" — https://arxiv.org/abs/2606.19116
  23. Same paper, HTML version (v1), section 4.6 "A New agents.txt Standard", checked 2026-10-29: "We propose agents.txt, a structured declaration file served at a well-known URL (/.well-known/agents.txt)" — https://arxiv.org/html/2606.19116v1
  24. Lago, /agents.txt on getlago.com, checked 2026-10-29: "# Standard: https://agents-txt.com" — https://getlago.com/agents.txt
  25. pg.ddx.io, /agents.txt, checked 2026-10-29: "# Standard: https://agents-txt.com" — https://pg.ddx.io/agents.txt
  26. General Informatics, /agents.txt on www.general-informatics.com, checked 2026-10-29: "# Standard: https://agents-txt.com" — https://www.general-informatics.com/agents.txt
  27. DFX Intel, /agents.txt on dfxintel.com, checked 2026-10-29: "# Standard: https://agents-txt.com" — https://dfxintel.com/agents.txt
  28. Algis Dumbris, MCPProxy Blog, The agents.txt Draft Just Expired, 2026-04-10, checked 2026-10-29: "Today, April 10, 2026, draft-srijal-agents-policy-00 quietly expired." — https://mcpproxy.app/blog/2026-04-10-agents-txt-expired/
  29. Kelson, K-Squared Ramblings, New agents.txt file found on DreamHost, 2026-05-13, checked 2026-10-29: "This morning I discovered that a new file had been added, agents.txt, to the root of each site, on May 7." "The first agents.txt spec I found used a completely different syntax and a completely different purpose." — https://journal.kvibber.com/2026/05/agents-txt-on-dreamhost/
  30. kvibber.com, /agents.txt (a DreamHost-hosted site), checked 2026-10-29: "Allow-Training: no Allow-RAG: yes Allow-Actions: no" — https://kvibber.com/agents.txt
  31. Jacob Campbell, BuiltToWinWeb, Agent Discovery Files Explained, 2026-10-05, checked 2026-10-29: "/agents.txt: a newer, less settled convention for stating, in plain text, what agents may do on a site and where its agent interfaces are. There are competing drafts; treat it as a human-readable pointer file rather than a contract." — https://built2winweb.com/blog/agent-discovery-files/

JSON · Markdown · OKF v0.2