agenticweb.wiki

AIPREF Vocabulary (A Vocabulary For Expressing AI Usage Preferences)

Last checked · 42 primary sources

This page is about the vocabulary Internet-Draft of the IETF AI Preferences (aipref) working group, draft-ietf-aipref-vocab, which defines categories of use (AI Training, AI Use, Search) and how a statement of preference allows or disallows each [1][7]. It does not say where such statements are published: that is the separate working-group draft draft-ietf-aipref-attach, which carries them in a Content-Usage HTTP header field and a Content-Usage rule in robots.txt [19]; see AIPREF Content-Usage Attachment. It is not robots.txt (the Robots Exclusion Protocol, RFC 9309) itself, which the attachment draft would update [19]. It is not Cloudflare's Content Signals Policy, which defines its own Content-Signal robots.txt line with the signals search, ai-input and ai-train [24]; see Content Signals Policy. It is also not Really Simple Licensing (RSL), the Text and Data Mining Reservation Protocol (TDMRep) or ai.txt, which are separate specifications.

Overview

The AIPREF vocabulary is the IETF working-group draft that names kinds of automated use of web content, so that a publisher can say whether each is allowed: "This document defines a vocabulary for expressing preferences regarding how digital assets are used by automated processing systems." [1]. The current draft has three categories, AI Training, AI Use and Search, where Search covers applications that direct users to the original location and overrides the other two categories [1]. It defines meaning and a short text form such as "train-ai=y, search=n" but no place to put it; the companion attachment draft carries the same string in robots.txt and in an HTTP header [1][19]. It is a statement of preference, not a control: "An entity that receives usage preferences has a choice whether to follow those preferences." [1].

Who uses it

Sites that serve a Content-Usage preference, checked live on 2026-10-28. Three of the four use the label "ai", which -01 defined [12] but -08 does not; -08 says "when processing a parsed Dictionary to obtain preferences, any unknown labels MUST be ignored." [1]. No Content-Usage line was found in the robots.txt files of www.ietf.org, www.mozilla.org, blog.cloudflare.com, en.wikipedia.org, www.nytimes.com or www.theguardian.com (checked 2026-10-28).

Who reads it

No AI crawler operator or AI vendor was found saying, with a date, that its crawlers read AIPREF Content-Usage preferences (checked 2026-10-28). Third-party guides say the same: "I could not find a primary source from a major crawler operator saying it reads Content-Usage, so treat support as unknown and uneven." (2026-10-11) [39]. The statements below are from Cloudflare, which runs its own Content Signals Policy, and do not say it reads Content-Usage.

Recent coverage

See also

References

  1. P. Keller (Open Future) and M. Thomson (Mozilla, editor), A Vocabulary For Expressing AI Usage Preferences, draft-ietf-aipref-vocab-08, Internet-Draft, 14 September 2026, checked 2026-10-28: "This document defines a vocabulary for expressing preferences regarding how digital assets are used by automated processing systems." "This vocabulary allows for the declaration of restrictions or permissions for use of digital assets by such systems." "Its contents DO NOT REFLECT CONSENSUS of the Working Group either in whole or part. Presense or absense of any particular text does not indicate consensus, and this document is published solely as a basis of further discussion." "This section does not yet have consensus" "Using an asset to modify the learned parameters of a generative AI model." "Using an asset as input to a generative AI model, where the asset is not directly provided by the user." "Using an asset in an application where the primary purpose of the application is to select assets and direct users to the location of those assets." "This category does not include the use of assets to generate summaries." "This category of use overrides any usage that falls into other categories, including AI Training (Section 4.1) and AI Use (Section 4.2)." "Extensions to this vocabulary are defined in a standards-track RFC that updates this document." "This process results in one of three potential answers: allow, disallow, and unknown." "This document takes no position on what default might be assigned." "This process ensures that the most restrictive preference applies." "For example, the following states a preference to allow model training (Section 4.1), disallow search (Section 4.3), with the preference for other categories being unknown: train-ai=y, search=n" "when processing a parsed Dictionary to obtain preferences, any unknown labels MUST be ignored." "An entity that receives usage preferences has a choice whether to follow those preferences." "Preferences are not a security mechanism." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-08.txt
  2. IETF Datatracker, draft-ietf-aipref-vocab document page, checked 2026-10-28: "Document Type Active Internet-Draft (aipref WG) Authors Paul Keller, Martin Thomson" "Replaces draft-keller-aipref-vocab" "Intended RFC status Proposed Standard" "WG state WG Document" "IESG state I-D Exists" "Associated WG milestone Aug 2026 A standard track specification describing vocabulary for expressing AI-related preferences to the IESG for publication" — https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/
  3. IETF Datatracker, draft-ietf-aipref-vocab history, checked 2026-10-28: "2025-04-30 00 Mark Nottingham WG -00 approved" "Mark Nottingham IETF WG state changed to In WG Last Call from WG Document" "Mark Nottingham IETF WG state changed to WG Document from In WG Last Call" — https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/history/
  4. IETF Datatracker API, draft-ietf-aipref-vocab document record (rev "08", time "2026-09-14T06:18:02Z", expires "2027-03-18T06:18:02Z"), checked 2026-10-28: "2026-09-14T06:18:02Z" — https://datatracker.ietf.org/api/v1/doc/document/draft-ietf-aipref-vocab/?format=json
  5. IETF Datatracker API, new-revision events for draft-ietf-aipref-vocab (posting times in UTC), checked 2026-10-28: "New version available: draft-ietf-aipref-vocab-08.txt" — https://datatracker.ietf.org/api/v1/doc/newrevisiondocevent/?doc__name=draft-ietf-aipref-vocab&format=json&limit=50
  6. IETF Datatracker API, documents of type RFC in group aipref (total_count 0), checked 2026-10-28: "total_count": 0 — https://datatracker.ietf.org/api/v1/doc/document/?group__acronym=aipref&type=rfc&format=json
  7. IETF Datatracker, AI Preferences (aipref) working group, About page with charter and milestones, checked 2026-10-28: "Area Web and Internet Transport (wit) State Active Charter charter-ietf-aipref-01 Approved" "Chairs Mark Nottingham, Suresh Krishnan Area Director Mike Bishop" "The AI Preferences Working Group will standardize building blocks that allow for the expression of preferences about how content is collected and processed for Artificial Intelligence (AI) model development, deployment, and use." "A standard track document covering vocabulary for expressing AI-related preferences, independent of how those preferences are associated with content." "Technical enforcement of preferences" "2026-08-31 Aug 2026 A standard track specification describing vocabulary for expressing AI-related preferences to the IESG for publication draft-ietf-aipref-vocab" — https://datatracker.ietf.org/wg/aipref/about/
  8. IETF Datatracker, aipref working group history, checked 2026-10-28: "2025-01-07 Cindy Morgan Charter approved, group active" "2025-09-23 Mike Bishop Changed milestone "A standard track specification describing vocabulary for expressing AI-related preferences to the IESG for publication", set due date to August 2026 from August" — https://datatracker.ietf.org/wg/aipref/history/
  9. IETF Datatracker, draft-keller-aipref-vocab, checked 2026-10-28: "Replaced Internet-Draft (candidate for aipref WG) Expired & archived Author Paul Keller Last updated 2025-04-10 (Latest revision 2025-03-30) Replaced by draft-ietf-aipref-vocab" — https://datatracker.ietf.org/doc/draft-keller-aipref-vocab/
  10. Mark Nottingham, [ai-control] Adoption of draft-keller-aipref-vocab-01, ai-control mailing list, 2025-04-26, checked 2026-10-28: "your chairs believe there is rough consensus to adopt draft-keller-aipref-vocab-01 as the starting point of our vocabulary work." "Accordingly, we appoint Paul Keller and Martin Thomson as editors." — https://mailarchive.ietf.org/arch/msg/ai-control/LuWDC12gjlQ6Yj5eKN7ZkgbxopA/
  11. P. Keller and M. Thomson, Proposal for an Opt-Out Vocabulary, draft-ietf-aipref-vocab-00, 1 May 2025, checked 2026-10-28: "This document proposes a standardized vocabulary of use cases that can be targeted when expressing machine-readable opt-outs related to Text and Data Mining (TDM) and AI training." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-00.txt
  12. draft-ietf-aipref-vocab-01 (category table: tdm, ai, genai, search, inference), checked 2026-10-28: "Table 1 tabulates this mapping." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-01.txt
  13. draft-ietf-aipref-vocab-02 (category table: all, train-ai, train-genai, ai-use, search), checked 2026-10-28: "Table 1 tabulates this mapping." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-02.txt
  14. draft-ietf-aipref-vocab-03 (category table: bots, train-ai, train-genai, search), checked 2026-10-28: "Table 1 tabulates this mapping." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-03.txt
  15. draft-ietf-aipref-vocab-04 (category table: bots, train-ai, ai-output, search), checked 2026-10-28: "Table 1 tabulates this mapping." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-04.txt
  16. draft-ietf-aipref-vocab-05 (category table: train-ai, search), checked 2026-10-28: "Table 1 tabulates this mapping." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-05.txt
  17. draft-ietf-aipref-vocab-06 (category table: train-ai, search), checked 2026-10-28: "Table 1 tabulates this mapping." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-06.txt
  18. draft-ietf-aipref-vocab-07 (category table: train-ai, search), checked 2026-10-28: "Table 1 tabulates this mapping." — https://www.ietf.org/archive/id/draft-ietf-aipref-vocab-07.txt
  19. G. Illyes (Google) and M. Thomson (Mozilla), Associating AI Usage Preferences with Content in HTTP, draft-ietf-aipref-attach-05, 19 August 2026, checked 2026-10-28: "Methods are defined for associating usage preferences with content that is obtained using the HTTP protocol. This document defines attachment methods using the Robots Exclusion Protocol and HTTP header fields." "This document updates RFC 9309 to allow for the inclusion of usage preferences." "The Content-Usage field is a structured field dictionary, as defined in Section 3.2 of [FIELDS]. This field follows the vocabulary and processing rules in [VOCAB]." — https://www.ietf.org/archive/id/draft-ietf-aipref-attach-05.txt
  20. IETF Datatracker, draft-ietf-aipref-attach document page, checked 2026-10-28: "Associating AI Usage Preferences with Content in HTTP" — https://datatracker.ietf.org/doc/draft-ietf-aipref-attach/
  21. IETF Datatracker, draft-ietf-aipref-attach history, checked 2026-10-28: "2026-05-01 04 (System) Document has expired" "2026-08-18 05 Martin Thomson New version available: draft-ietf-aipref-attach-05.txt" — https://datatracker.ietf.org/doc/draft-ietf-aipref-attach/history/
  22. GitHub, ietf-wg-aipref/drafts repository, checked 2026-10-28: "AIPREF working drafts" — https://github.com/ietf-wg-aipref/drafts
  23. Editor's copy of the vocabulary draft on GitHub Pages, checked 2026-10-28: "A Vocabulary For Expressing AI Usage Preferences" — https://ietf-wg-aipref.github.io/drafts/draft-ietf-aipref-vocab.html
  24. Will Allen, Cloudflare Blog, Giving users choice with Cloudflare's new Content Signals Policy, 2025-09-24, checked 2026-10-28: "This policy defines three content signals - search, ai-input, and ai-train - and their relevance to crawlers." — https://blog.cloudflare.com/content-signals-policy/
  25. Michael Tremante, [ai-control] Cloudflare's Content Signal Policy, ai-control mailing list, 2025-09-24, checked 2026-10-28: "We see this as an interim measure to address immediate needs, while work is done towards standardization." "We will continue to engage in this working group and hope we can achieve a consensus position and will revisit our policy once that has happened." — https://mailarchive.ietf.org/arch/msg/ai-control/EANb9b9bljOanLBFm5FRerYlxyI/
  26. M. Tremante and L. Romm, draft-romm-aipref-contentsignals-00, individual Internet-Draft, 2025-10-01, checked 2026-10-28: "The proposal is for these categories to nest within the larger category of Automated Processing" — https://datatracker.ietf.org/doc/html/draft-romm-aipref-contentsignals-00
  27. IETF Datatracker, draft-romm-aipref-contentsignals, checked 2026-10-28: "Expired Internet-Draft (individual) Expired & archived Authors Michael Tremante, Leah Romm" — https://datatracker.ietf.org/doc/draft-romm-aipref-contentsignals/
  28. Bryan Becker, Cloudflare Blog, Have it both ways: stay discoverable in search while disallowing AI training, 2026-09-15, checked 2026-10-28: "As standards such as ai-prefs mature, we will revisit this approach." "Open standards such as ai-prefs will be an important part of making that possible." — https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/
  29. Cloudflare press release, Cloudflare Helps End the Search-or-AI-Training Tradeoff, 2026-09-15, checked 2026-10-28: "Cloudflare is also an active participant in open standards work at the Internet Engineering Task Force (IETF), including "ai-prefs," a specification under development that would allow any website to express its AI access preferences in a standardized, portable way." "Today's controls are designed to work alongside these emerging standards." — https://www.cloudflare.com/press/press-releases/2026/cloudflare-helps-end-the-search-or-ai-training-tradeoff/
  30. contentsignals.org, site description (meta description, undated), checked 2026-10-28: "An up-to-date guide to the IETF's proposed new AI Preferences (aipref): a new way for website publishers to control how automated systems use their content." — https://contentsignals.org/
  31. SpinXO, robots.txt, checked 2026-10-28: "AI usage preference (draft-ietf-aipref-attach)." — https://www.spinxo.com/robots.txt
  32. SpinXO, home page HTTP response header, checked 2026-10-28: "content-usage: train-ai=y, search=y" — https://www.spinxo.com/
  33. Launchpad, robots.txt, checked 2026-10-28: "Content-Usage: ai=n" — https://launchpad.net/robots.txt
  34. Launchpad Bugs, robots.txt, checked 2026-10-28: "Content-Usage: ai=n" — https://bugs.launchpad.net/robots.txt
  35. Launchpad Code, robots.txt, checked 2026-10-28: "Content-Usage: ai=n" — https://code.launchpad.net/robots.txt
  36. Launchpad Answers, robots.txt, checked 2026-10-28: "Content-Usage: ai=n" — https://answers.launchpad.net/robots.txt
  37. Ubuntu Changelogs, robots.txt, checked 2026-10-28: "Content-Usage: ai=n" — https://changelogs.ubuntu.com/robots.txt
  38. They Said So, robots.txt, checked 2026-10-28: "Content-Usage: ai=n" — https://theysaidso.com/robots.txt
  39. IMozz, ai SiteReady, Content-Usage (aipref): robots.txt and HTTP header snippets, updated 2026-10-11, checked 2026-10-28: "I could not find a primary source from a major crawler operator saying it reads Content-Usage, so treat support as unknown and uneven." — https://www.ai-siteready.com/blog/content-usage-aipref-robots-txt-http
  40. Mark Nottingham and Suresh Krishnan, IETF Blog, Enabling Publishers to Express Preferences for AI Crawlers: An Update on the AIPREF Working Group, 2025-10-30, checked 2026-10-28: "Accordingly, we made a Working Group Last Call on the documents in September." "The response was not one we anticipated: several participants raised fundamental issues about the nature of the Internet-Drafts (I-Ds), forcing the group to reconsider its approach." — https://www.ietf.org/blog/ai-pref-update/
  41. AI Crawler Check, Content-Usage in robots.txt: The IETF AIPREF Standard Explained, 2026-10-18, checked 2026-10-28: "No major AI company has said publicly that its crawler reads Content-Usage yet, while every major one documents its tokens." — https://aicrawlercheck.com/blog/ietf-aipref-content-usage-robots-txt
  42. Will Schweitzer, The Scholarly Kitchen, Guest Post — All this AI Signal and Noise: What Deserves Action Now, 2026-10-22, checked 2026-10-28: "On September 13, the AI preferences working group published a vocabulary with three verbs: training, use, and search." — https://scholarlykitchen.sspnet.org/2026/10/22/guest-post-all-this-ai-signal-and-noise-what-deserves-action-now/

JSON · Markdown · OKF v0.2