Opens in a new tabSkip to content
Agent LighthouseAgent Lighthouse

    Searches the text of every published page. The evidence sources themselves are not in this index — search all of them on the trusted sources page.

    GitHub ↗
    Browse checks and page contents
    access-crawl-control/no-nofollow

    No nofollow on important links

    What it checks

    A site-wide nofollow directive prevents AI crawlers from following links to discover your content. Important internal links should be followable.

    Why it matters

    A page-level <meta name="robots" content="nofollow"> (or ‘none’) stops documented AI-serving crawlers from following any link on that page; per-link rel="nofollow" is treated by Google as a hint that links ‘will generally not be followed’. Applying either to internal navigation therefore reduces discovery of the linked pages. Falsifiable: if pages linked only from nofollowed internal links are crawled and indexed at the same rate as normally linked pages, the claim fails.

    Evidence

    Apple’s current Applebot documentation is unambiguous and names the behavior directly: among supported robots meta directives it lists ‘nofollow: Applebot won’t follow any links on the page’ and ‘none: Applebot won’t index, snippet, or follow links on the page’. Applebot also supports these via the X-Robots-Tag HTTP header, and falls back to Googlebot’s robots.txt rules when Applebot is not named — so Google-targeted directives leak into Apple’s AI-grounding crawl. Google’s own link-qualification documentation states that links marked nofollow, sponsored or ugc ‘will generally not be followed’.

    Since Google’s AI features require the target page to be ‘indexed and eligible to be shown in Google Search with a snippet’, suppressing traversal to a page suppresses its AI eligibility by the same chain established in signal 1. Two major vendors documenting the consumer behavior in their own crawler docs is what grade A requires.

    Limits

    Two significant qualifications. First, Apple’s documented nofollow is the PAGE-LEVEL meta robots directive, not per-link rel="nofollow" — most audits conflate these, and Apple’s docs say nothing about the rel attribute on individual anchors. An audit that flags a single rel="nofollow" internal link cannot cite the Applebot page as support. Second, Google demoted nofollow from directive to hint in September 2019. It states plainly that ‘the linked pages may be found through other means, such as sitemaps or links from other sites, and thus they may still be crawled’.

    So nofollow does not reliably prevent discovery even for Google. OpenAI, Anthropic and Perplexity documentation is entirely silent on nofollow, with no evidence GPTBot, ClaudeBot or PerplexityBot honors it in either form. Scope the audit to meta robots nofollow/none on indexable pages, and treat per-link rel=nofollow on internal navigation as a weaker informational finding.

    How it scores

    Apple’s Applebot documentation names the behaviour directly. Among the supported robots meta directives it lists “nofollow: Applebot won’t follow any links on the page” and “none: Applebot won’t index, snippet, or follow links on the page”. Both are honoured in the X-Robots-Tag header as well. A named agent, a named directive and a stated consequence is grade A. The grade covers the page-level directive only. Apple says nothing about rel="nofollow" on individual anchors, and Google treats that attribute as a hint whose links “will generally not be followed” — so flagging a single per-link nofollow cannot cite this evidence, and the audit does not.

    Sources