Opens in a new tabSkip to content
Agent LighthouseAgent Lighthouse

    Searches the text of every published page. The evidence sources themselves are not in this index — search all of them on the trusted sources page.

    GitHub ↗
    Browse checks and page contents
    access-crawl-control/chatgpt-user

    ChatGPT-User allowed

    What it checks

    Without an explicit robots.txt rule, ChatGPT-User may still crawl your site but has no signal that it is welcome. Adding an explicit allow rule improves your visibility in AI-powered search and ensures consistent crawler behavior.

    Why it matters

    Disallowing ChatGPT-User is intended to stop user-initiated ChatGPT fetches of the site. But OpenAI reserves an exemption, and field measurement shows the disallow is frequently not honored. The directive’s presence therefore does not reliably predict agent behavior in either direction.

    Evidence

    ChatGPT-User allow/block state in robots.txt

    OpenAI documents the agent (UA ‘ChatGPT-User/1.0; +https://openai.com/bot’, IPs at openai.com/chatgpt-user.json) as handling ‘user-initiated actions in ChatGPT and Custom GPTs’. It is the dominant user-action agent by volume: Cloudflare Radar attributes ‘nearly three quarters of the request traffic’ in the user-action category to ChatGPT-User (July 2025), and 14.9% of News & Publications AI traffic.

    Limits

    Two independent refutations of the block mechanism. (1) Vendor exemption, stated verbatim: ‘Because these actions are initiated by a user, robots.txt rules may not apply.’ (2) Field measurement. TollBit’s H1 2026 ‘State of the Bots’ found that ChatGPT-User ‘reached disallowed pages on more sites than any other bot’. It also found that ChatGPT-User, Bytespider and Youbot ‘each accessed disallowed pages on nearly half of the European sites that had explicitly listed them’. Blocking also costs visibility: BuzzStream found the lowest citation retention of any bot studied (70.6%) among sites blocking ChatGPT-User. Do not score a ChatGPT-User disallow as a positive; report it as informative with this caveat.

    Sources