Opens in a new tabSkip to content
Agent LighthouseAgent Lighthouse

    Searches the text of every published page. The evidence sources themselves are not in this index — search all of them on the trusted sources page.

    GitHub ↗
    Browse checks and page contents
    machine-discovery/cors-ai-files

    CORS on AI files

    What it checks

    Whether each published AI file — /llms.txt, /.well-known/ard.json and /.well-known/ai-catalog.json — answers a cross-origin request with Access-Control-Allow-Origin. Only files the scan’s root fetch found present are probed. Only agent code running inside a browser origin needs that header. Server-side crawlers, agent backends and MCP clients are not browsers and read the files either way.

    The audit sends an OPTIONS preflight with Origin and Access-Control-Request-Method: GET, then a GET with Origin if the preflight carries no header. Either answer with the header counts.

    Why it matters

    CORS governs requests that scripts make inside a browser (MDN, verified 2026-10-18). AI crawlers and assistants fetch llms.txt and the AI catalog from servers, where CORS does not apply. No AI vendor documents fetching these files from a browser. The one documented browser-side case, an MCP Apps widget in a sandboxed iframe (MCP Apps CSP and CORS, verified 2026-10-18), calls the author’s own APIs; agent-interfaces/cors-api-routes covers that.

    Evidence

    CORS headers on public AI files and API routes

    Mechanism: The claim under test: serving Access-Control-Allow-Origin (typically ‘*’) on llms.txt, .md mirrors, feeds and public JSON endpoints is required for AI agents to fetch and use them.

    The mechanism is real but its scope is much narrower than the audit implies. CORS matters only for code running inside a browser origin: MDN states ‘browsers restrict cross-origin HTTP requests initiated from scripts’, and the server merely opts in via ACAO. The genuine AI consumer class is browser-sandboxed agent code. OpenAI’s Apps SDK widgets run in an isolated iframe under a strict CSP, and must declare connect_domains — mapped to connect-src — for every origin they will fetch from. Such a widget fetching a publisher’s JSON or llms.txt cross-origin will be blocked without ACAO. That class is small today but growing.

    Limits

    Decisive counter-evidence for the general case: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot and every server-side agent backend are non-browser HTTP clients. MDN states that browsers restrict cross-origin requests from scripts; it is this project’s inference, not MDN’s statement, that these clients therefore do not implement the same-origin policy and are completely unaffected by a missing Access-Control-Allow-Origin header. Browser extensions with host permissions (the Claude-in-Chrome / sidebar class) also bypass CORS. No AI vendor doc requires CORS on publisher resources, and the llms.txt spec says nothing about it. Therefore ‘missing CORS blocks AI agents’ is false as a general claim. Recommend rewording the audit to target only browser-embedded agent consumption and demoting it out of the score.

    How it scores

    Informative, weight 0. Every result carries an unconsumed notice. The audit never warns or fails. No published AI file is not applicable, and nothing is probed. Files with or without an Access-Control-Allow-Origin header pass; files without one are named, with the note that crawlers and server-side agents do not need it.

    Sources