mergent.co agentic web readiness

B Manifest validity: 100% (1/1 valid, 0 errors, 3 warnings) · Surface score 33 (coverage 20 + breadth 3 + conformance 10). 3 standards observed across 2 of 9 steps; 1 manifest present, 0 errors, 3 warnings. Checked 2026-10-25 23:45:56 UTC. This grades the agent-facing files published at mergent.co, not the organisation's products.

Badge for a README: ![agentic web grade](https://agenticweb.wiki/badge/mergent.co.svg) · JSON · Markdown · Re-check now

Coverage by step

Manifests

What an agent can call

• robots.txt AI policy: blocks GPTBot, ClaudeBot, Claude-Web, anthropic-ai, Google-Extended, CCBot, Applebot-Extended, meta-externalagent, Bytespider, Amazonbot, cohere-ai, Diffbot; unlisted crawlers allowed

Standards observed

• Content Signals Policy https://mergent.co/robots.txt
• AI Crawler User-Agent Tokens in robots.txt https://mergent.co/robots.txt
• Markdown for Agents https://mergent.co/

Cheapest next steps

1. llms.txt: one Markdown index file; the most widely read agent-facing file
   Generate starter /llms.txt for mergent.co

   # Mergent
   
   > Mergent and what it offers to agents
   
   Short notes an agent should know before using the site: what it is for, who runs it, how to cite it.
   
   ## Docs
   - [Getting started](https://mergent.co/docs/getting-started): the first page to read
   - [API reference](https://mergent.co/openapi.json): OpenAPI description
   
   ## Optional
   - [Changelog](https://mergent.co/changelog): what changed, newest first
   

2. Agentic Resource Discovery: a catalog at /.well-known/ard.json lets agents discover your APIs, MCP servers and data in one fetch (emerging spec — publish only if you run the underlying endpoints; generate via MCP template({"id": "ard", "domain": "mergent.co"}))
3. AGENTS.md: instructions for coding agents working on or with the site
   Generate starter /AGENTS.md for mergent.co

   # Mergent for agents
   
   Mergent and what it offers to agents
   
   ## Read the data, not the HTML
   - Index: https://mergent.co/llms.txt
   - API description: https://mergent.co/openapi.json
   - Discovery catalog: https://mergent.co/.well-known/ard.json
   
   ## Calling
   - MCP server (Streamable HTTP): https://mergent.co/mcp
   - Rate limits and authentication: describe them here
   
   ## Conventions
   - How to cite or attribute content
   - What not to do (no scraping of /account/, no automated sign-ups, etc.)
   
   Contact: security@mergent.co
   

Agent pre-flight & practical readiness

Preferred mode: markdown_negotiation. GET / with Accept: text/markdown. The origin already serves a Markdown body for agents. mergent.co publishes no root agent discovery manifests, but docs.mergent.co serves /llms.txt (48 links). Probe docs.mergent.co or publish /.well-known/ard.json on mergent.co to link them.

Fetch: GET https://mergent.co/ GET / with Accept: text/markdown. The origin negotiates a Markdown body.

Subdomain discovery hint (subdomain_stranded): mergent.co publishes no root agent discovery manifests, but docs.mergent.co serves /llms.txt (48 links). Probe docs.mergent.co or publish /.well-known/ard.json on mergent.co to link them. (check docs.mergent.co)

Practical readability: exa_parsable yes, markdown negotiation yes, render markdown_native, HTML quality high, first content at byte 0.

AI crawler posture: search_allowed_training_blocked · Assistant access: OpenAI allowed, Anthropic allowed, Google blocked, Perplexity allowed, Exa allowed.

Readiness subscores: reach 20/20, readability 30/30, capabilities 2/35, conformance 10/15, total 62/100, grade B.

How the grade is made

+10 per lifecycle step with a standard observed, counting manifests that validate or that the probe rule for the standard recognises (a valid MCP server card counts for MCP; max 90); +3 per observed standard beyond the first two (max 15); conformance +15 (+10 with one manifest) minus 5 per agent-facing manifest with errors, never below 0 (security.txt is reported, not scored). Lifecycle grade: A+ >= 60, A >= 45, B >= 30, C >= 20, D >= 10. Strict 100-point composite (subscores.total_100 / subscores.grade_100): reach_and_policy (20) + readability (30) + capabilities (35) + conformance (15), where A+ >= 80, A >= 65, B >= 45, C >= 28, D >= 15. Errors are specification violations; warnings are best-practice gaps. Only public paths are fetched, without credentials, so standards that need a login (skipped in the probe) are not counted either way. The same report is the validate_manifest tool on the MCP server.

JSON · Markdown · probe rules