Opens in a new tabSkip to content
Agent LighthouseAgent Lighthouse

    Searches the text of every published page. The evidence sources themselves are not in this index — search all of them on the trusted sources page.

    GitHub ↗
    Browse checks and page contents
    access-crawl-control/duckassistbot

    DuckAssistBot allowed

    What it checks

    Without an explicit robots.txt rule, DuckAssistBot may still crawl your site but has no signal that it is welcome. Adding an explicit allow rule improves your visibility in AI-powered search and ensures consistent crawler behavior.

    Why it matters

    Disallowing DuckAssistBot removes the site as a real-time source for DuckDuckGo’s AI-assisted answers (effective after ~72 hours) without affecting organic DuckDuckGo search rankings; the crawl is documented as never used for model training.

    Evidence

    DuckAssistBot allow/block state in robots.txt

    DuckDuckGo publishes an unusually precise help page. It names the token, ‘DuckAssistBot/1.2; (+http://duckduckgo.com/duckassistbot.html)’, and the purpose: ‘DuckAssistBot is a web crawler for DuckDuckGo Search that crawls pages in real-time for our AI-assisted answers’. It rules out training use: ‘This data is not used in any way to train AI models’. It states the latency of an opt-out. A disallow ‘will take effect after 72 hours and DuckAssistBot will stop crawling your site’. It also gives a decoupling guarantee: ‘Opting out of DuckAssistBot does not impact organic search rankings.’ All three audit-relevant facts — compliance, latency and non-training use — are vendor-stated and falsifiable.

    Limits

    DuckAssistBot does not appear in Cloudflare Radar’s named AI-crawler top-five breakdowns, so its traffic volume — and therefore the practical stakes of allowing or blocking it — is small relative to GPTBot/ClaudeBot/ChatGPT-User. The 72-hour enforcement lag means a disallow is not immediate.

    How it scores

    DuckDuckGo’s help page is unusually precise. It publishes the token, and states that the crawler “crawls pages in real-time for our AI-assisted answers” and that “This data is not used in any way to train AI models”. It even names the enforcement lag: a disallow takes effect after roughly 72 hours. A vendor documenting the token, the use and the timing is well past the grade-A bar. The practical stakes are small, since the bot does not appear in Cloudflare Radar’s named top-five breakdowns, but that is a question of volume rather than of evidence.

    Sources