Opens in a new tabSkip to content
Agent LighthouseAgent Lighthouse

    Searches the text of every published page. The evidence sources themselves are not in this index — search all of them on the trusted sources page.

    GitHub ↗
    Browse checks and page contents
    content-extraction/markdown-alternate

    Markdown alternate: resolvable, faithful, cheaper

    What it checks

    Where a site serves a markdown version of a page, this checks that the markdown is usable: that it resolves as text/markdown, still carries the page’s headings and prose, and costs fewer tokens than the HTML. It looks on three routes — a declared <link rel="alternate" type="text/markdown">, the page URL plus .md, and the page URL with Accept: text/markdown.

    Interactive coding agents read markdown when a site offers it, and it costs them far fewer tokens than the HTML. A markdown version that has drifted from the page is worse than none: the agent gets a document that looks authoritative, costs less, and says less than the page it claims to mirror.

    A site that serves no markdown version is reported as not applicable rather than failed. The consumers documented for this mechanism are coding agents, and no source measures a cost to a site that offers no markdown at all.

    Why it matters

    HTML <link> tags pointing to llms.txt — Emitting <link rel="describedby" href="/llms.txt"> and/or <link rel="alternate" type="text/markdown" href="...md"> in the HTML head (or the equivalent HTTP Link: header) causes agents to discover and fetch those resources.

    Markdown alternate representations of pages — Serving a markdown representation of each page — via HTTP content negotiation on Accept: text/markdown and/or a .md URL suffix — causes named AI coding agents to retrieve markdown instead of HTML, substantially reducing tokens consumed per page.

    Evidence

    The relations are real and specified — but only since 2026-08-10, and the two halves differ sharply in strength. The llms.txt v2 spec explicitly defines both relations. rel="alternate" type="text/markdown" points to the markdown version of a page, and rel="describedby" points to the covering llms.txt. Either can be delivered as an HTML <link> element or as an HTTP Link: header; the header form also works for non-HTML resources, and is configurable at CDN level.

    This was the headline addition in v2. Deployment exists: a live check found developers.cloudflare.com emits <link rel="alternate" type="text/markdown" href="[developers.cloudflare.com/…/index.md](https://developers.cloudflare.com/fundamentals/index.md)">, and Mintlify advertises resource locations via HTTP Link headers across every site it hosts. For the markdown half there is one named consumer: acceptmarkdown.com’s June 2026 matrix reports OpenAI’s Codex CLI fetches HTML first, then ‘parses the response for <link rel="alternate" type="text/markdown" href=…>’ and requests the markdown version separately. Note that any audit checking a bespoke rel="llms-txt" or rel="llms" value would be invented — only describedby and alternate are spec’d.

    Markdown alternate representations of pages

    The only grade-A signal in this domain, and it is the one that should carry weight in the audit. Anthropic documents the behavior explicitly for a named agent: ‘WebFetch sets a User-Agent header beginning with Claude-User, and an Accept header that prefers Markdown over HTML so servers that support content negotiation can return Markdown directly.’ Cloudflare shipped Markdown for Agents in Feb 2026. It does network-level HTML-to-markdown conversion on Accept: text/markdown, and returns an x-markdown-tokens header and YAML frontmatter.

    Independent header-capture testing finds seven agents advertising text/markdown as of June 2026 (Claude Code, Copilot Chat, Copilot CLI, Cursor, Microsoft Copilot, OpenClaw, OpenCode), up from three in February — a clear upward trajectory. Field measurement confirms real volume: Evil Martians recorded ~40,000 markdown fetches, 15% of all agent reads across 268k requests, with Claude Code at 76% markdown; Dries Buytaert measured GPTBot taking markdown 34.8% of the time via .md URLs.

    A live check confirmed the mechanism: curl -H ‘Accept: text/markdown’ returns content-type: text/markdown from Anthropic’s, Stripe’s and Cloudflare’s docs today. This mechanism also needs no site-specific knowledge from the agent, unlike llms.txt.

    Limits

    HTML <link> tags pointing to llms.txt — The rel="describedby" -> llms.txt half has no known consumer at all. Decisively, Lighthouse’s own gatherer (source verified) resolves new URL(‘/llms.txt’, finalDisplayedUrl) and nothing else — it ignores link tags entirely, so even Google’s llms.txt tooling would never see the tag. There is no auto-discovery-links audit in Lighthouse’s Agentic Browsing category; fetching that hypothesised URL returns HTTP 404, and the category index lists only seven audits, none of them about link relations.

    The Codex CLI claim is single-sourced and it could not be independently corroborated; Checkly’s Feb 2026 testing found Codex sending no markdown preference at all. Adoption is near-zero and unmeasured: of three sites sampled, only Cloudflare emitted the tag; Stripe and Next.js emitted none, and Stripe instead uses an unrelated Link: rel="service-meta" pointing at /.well-known/skills/index.json. The relations are two weeks old as of this research.

    Markdown alternate representations of pages — Grade A applies to interactive coding agents, not to search crawlers or consumer chat — audits should say so. ChatGPT-User takes markdown on just 0.1% of fetches; ChatGPT, Claude.ai, Perplexity, Gemini variants, Grok, Windsurf, Devin, Aider, Cline, v0 and Zed send HTML-only Accept headers. Otterly’s 14-day controlled test found 0 crawler visits and 0 citations for .md files versus 137 visits to matched HTML, and Longato’s 24-hour two-site CDN check found zero .md requests from GPTBot/ClaudeBot/PerplexityBot.

    The two best studies also directly contradict each other on which mechanism works: Dries (Jan 2026) states ‘No AI crawler uses content negotiation. Not one’ and saw markdown taken only via .md URLs. Evil Martians (May-Jul 2026) concluded instead that content negotiation ‘is the mechanism actually delivering Markdown to the client that wants it (Claude Code)’. The later date and the crawlers-versus-agents population reconcile the two, and both argue for supporting .md URLs and the Accept header rather than choosing between them.

    Google states you don’t need Markdown to appear in Search or its AI features. Dries also found serving markdown did not reduce bot load; crawl volume rose ~7%.

    How it scores

    HTML <link> tags pointing to llms.txt — The relations are real and specified — llms.txt v2 (2026-08-10) defines rel="alternate" type="text/markdown" for a page’s markdown version and rel="describedby" for the covering llms.txt, deliverable as <link> elements or as an HTTP Link: header. What no source shows is a consumer. Google’s own llms.txt tooling resolves new URL('/llms.txt', finalDisplayedUrl) and ignores link tags entirely, so even the one shipping checker would never see the tag. A published specification with no reader is exactly grade C: plausible, cheap, and unproven.

    Markdown alternate representations of pages — The one grade-A signal here, and it is vendor-stated for a named agent. “WebFetch sets a User-Agent header beginning with Claude-User, and an Accept header that prefers Markdown over HTML so servers that support content negotiation can return Markdown”. A named consumer, a named header and a stated behaviour is the grade-A bar. The scope is narrow on purpose. It covers interactive coding agents, not search crawlers or consumer chat: ChatGPT-User takes markdown on 0.1% of fetches, and ChatGPT, Perplexity, Gemini, Grok and the rest send HTML-only Accept headers.

    Sources