Really Simple Licensing RSL · /robots.txt

Overview

The spec bundles four pieces: the core RSL XML vocabulary for expressing usage, licensing, payment, and legal rules; the Open License Protocol (OLP), an OAuth 2.0 extension for acquiring and validating an RSL license at request time; the Crawler Authorization Protocol (CAP), an HTTP authorization scheme for crawlers holding a license; and the Encrypted Media Standard (EMS), a format for encrypting paywalled assets so unlicensed reuse is technically, not just contractually, blocked ^[1]. Discovery works through robots.txt references, HTML <link> elements, RSS entries, or HTTP files, pointing to an RSL XML license document; an example in the spec shows a publisher prohibiting all AI use (<prohibits type="usage">ai-all</prohibits>) while still allowing search indexing ^[1].

It does not itself provide agent identity or content delivery; those integrate via OLP/CAP with whatever identity and hosting stack a publisher already runs. It is explicitly an industry specification governed by a vendor consortium (the RSL Collective), not a submission to IETF or W3C — its own document header labels it Category 'Industry Specification', Status 'Recommendation' in RSL's own maturity scale, which should not be read as a W3C/IETF Recommendation.

As of this verification (2026-09-06), RSL 1.0 is current, superseding a 0.9 version referenced from the spec page, with an active GitHub issue tracker and a published errata page — signs of a maintained, living specification rather than a one-off announcement ^[1]. Given the roster of major media-industry editors (Condé Nast, Ziff Davis, Yahoo, O'Reilly), implementation is recorded as shipping on the publisher side, though this pass did not run a probe to count adoption, so the adoption level is unmeasured rather than claimed or measured.

What is unresolved: how many publishers have actually deployed an rsl.xml file versus simply being named as spec editors; whether any AI vendor has implemented the OLP client side to actually acquire and pay for a licensed RSL grant; and how RSL's paid-licensing model will coexist with the free, binary-only permission signals of AIPREF and Cloudflare's Content Signals Policy, which address the same robots.txt real estate with a simpler yes/no vocabulary.

Discovery and probe

robots.txt reference, HTML <link>, RSS/HTTP header, or a standalone rsl.xml file · /rsl.xml

GET {origin}/robots.txt
parse:     Look for an RSL reference line in robots.txt (pointing to an rsl.xml license file), or an HTML <link> element with an RSL relation, or a Content-License/Link HTTP header; then GET the referenced XML document and parse the <rsl> root element for <license>, <prohibits>, <permits> or <payment> children
on absent: No RSL license file is referenced from any of robots.txt, HTML, RSS, or HTTP headers; the site has not published RSL terms

Live check run by the MCP probe: GET /robots.txt. RSL is referenced from robots.txt by a License line or an rsl.xml link.

{
 "path": "/robots.txt",
 "expect": {
  "text_match": "(?im)^license:\\s*\\S|rsl\\.xml"
 },
 "note": "RSL is referenced from robots.txt by a License line or an rsl.xml link."
}

Instances

• The Guardian (theguardian.com) · Observed 2026-10-23
• rslstandard.org · Observed 2026-10-23
• medium.com · Observed 2026-10-23
• TollBit Content Licensing and Bot Paywall Clearinghouse · Observed 2026-07-15 · registry / facilitator
• RSL Collective (Really Simple Licensing) · Observed 2026-06-24 · registry / facilitator

Every instance with its source is listed on the instances page.

See also

• Builds on: AI crawler tokens in robots.txt
• Pairs with: AI crawler tokens in robots.txt, AIPREF Vocabulary (A Vocabulary For Expressing AI Usage Preferences), AIPREF Content-Usage Attachment, Content Signals Policy, Cloudflare Pay Per Crawl, TDMRep (TDM Reservation Protocol)
• Scanners & registries: Agents Welcome — Agent Protocol Atlas

Comparisons

• robots.txt (Allow/Disallow) vs Content-Signal, AIPREF & TDMRep (classic vs agentic)
• Really Simple Licensing vs AIPREF Vocabulary (A Vocabulary For Expressing AI Usage Preferences)
• AI crawler tokens in robots.txt vs Really Simple Licensing
• Really Simple Licensing vs TDMRep (TDM Reservation Protocol)
• Really Simple Licensing vs Content Signals Policy
• Really Simple Licensing vs CoMP (Content Monetization Protocols)

References

1. The RSL 1.0 specification, Document RSL-SPEC-1.0, Category 'Industry Specification', Status 'Recommendation', was published 2025-12-10 at the canonical URI/namespace https://rslstandard.org/rsl, with editors from RSL Collective, Condé Nast, Ziff Davis, Schema.org, Yahoo, Automattic, O'Reilly Media, and Fastly, and an issue tracker at github.com/rslstandard/rsl. — https://rslstandard.org/rsl (2025-12-10) VERIFIED
2. The rslstandard/rsl GitHub repository's README independently describes RSL as an open XML-based standard maintained by the RSL Technical Steering Committee, corroborating the spec page's own claims about scope and governance from a second, separately-hosted source, and states the specification's status as 'draft' at the repository level. — https://github.com/rslstandard/rsl VERIFIED

Verification log and errata

• 2026-09-06 — created and verified against rslstandard.org/rsl