crewai.com agentic web readiness
A Score 54 (coverage 30 + breadth 9 + conformance 15). 5 standards observed across 3 of 9 steps; 4 manifests present, 0 errors, 2 warnings. Checked 2026-10-20 02:03:49 UTC. This grades the agent-facing files published at crewai.com, not the organisation's products.
Coverage by step
Manifests
| Manifest | Result | Findings |
|---|---|---|
Agentic Resource Discovery/.well-known/ard.json | absent HTTP 404 | |
AI Catalog well-known file/.well-known/ai-catalog.json | absent HTTP 404 | |
MCP Server Cards (SEP-2127)/mcp/server-card | absent HTTP 404 | |
Agent2Agent Protocol/.well-known/agent-card.json | absent HTTP 404 | |
llms.txt/llms.txt | valid HTTP 200 | |
AGENTS.md/AGENTS.md | absent HTTP 404 | |
api-catalog: A Well-Known URI and Link Relation to Help Discovery of APIs/.well-known/api-catalog | valid HTTP 200 | |
AI Crawler User-Agent Tokens in robots.txt/robots.txt | valid HTTP 200 |
|
security.txt (RFC 9116)/.well-known/security.txt | valid HTTP 200 | |
auth.md/auth.md | absent HTTP 404 |
What an agent can call
- API description
https://crewai.com/openapi.json(application/vnd.oai.openapi+json;version=3.1) - API description
https://crewai.com/(text/html) - llms.txt: CrewAI, 27 links
- robots.txt AI policy: unlisted crawlers partial
- OAuth protected resource
https://crewai.com, authorization servers: https://crewai.com
Standards observed
- llms.txt https://crewai.com/llms.txt
- Content Signals Policy https://crewai.com/robots.txt
- api-catalog: A Well-Known URI and Link Relation to Help Discovery of APIs https://crewai.com/.well-known/api-catalog
- OAuth Protected Resource and Authorization Server Metadata (as used for agent authorization) https://crewai.com/.well-known/oauth-protected-resource
- AI Crawler User-Agent Tokens in robots.txt
Cheapest next steps
- Agentic Resource Discovery: a catalog at /.well-known/ard.json lets agents discover your APIs, MCP servers and data in one fetch
- AGENTS.md: instructions for coding agents working on or with the site
- MCP Server Cards (SEP-2127): if you run an MCP server, a SEP-2127 card at <mcp-url>/server-card lets clients configure themselves
How the grade is made
+10 per lifecycle step with a standard observed, counting manifests that validate or that the probe rule for the standard recognises (a valid MCP server card counts for MCP; max 90); +3 per observed standard beyond the first two (max 15); conformance +15 (+10 with one manifest) minus 5 per agent-facing manifest with errors, never below 0 (security.txt is reported, not scored). A+ >= 60, A >= 45, B >= 30, C >= 20, D >= 10. Errors are specification violations; warnings are best-practice gaps. Only public paths are fetched, without credentials, so standards that need a login (skipped in the probe) are not counted either way. The same report is the validate_manifest tool on the MCP server.
JSON · Markdown · probe rules