Opens in a new tabSkip to content
Agent LighthouseAgent Lighthouse

    Searches the text of every published page. The evidence sources themselves are not in this index — search all of them on the trusted sources page.

    GitHub ↗
    Browse checks and page contents
    machine-discovery/rss-feed-content

    RSS feed content complete

    What it checks

    How many items in the site’s first RSS or Atom feed carry more than 500 characters in content:encoded, description or content.

    Why it matters

    Feeds are read for URLs, not article bodies. Google (verified 2026-10-18) accepts RSS and Atom feeds as sitemaps, and says such a feed “only provides information on recent URLs”. The Common Crawl news crawler (verified 2026-10-18) takes links from feeds and fetches the article pages. No search engine or AI vendor documents reading full article text from a feed. Excerpt feeds are valid RSS and Atom. The feed’s discovery role is graded separately by machine-discovery/rss-feed.

    Evidence

    Mechanism claim: A crawler that fetches a site’s RSS or Atom feed ingests each item’s <content:encoded> or <atom:content> as the article body, and therefore does not request the article page. When items carry only a truncated excerpt, the crawler must fetch every article individually.

    Grade: C — full-text feed content is a published, widely implemented convention with real consumers in feed readers. But the only vendor that documents machine consumption of feeds is Google, and it documents reading them for URLs rather than for article bodies. No AI vendor documents ingesting article text from a feed.

    • The RSS 1.0 content module defines content:encoded as “An element whose contents are the entity-encoded or CDATA-escaped version of the content of the item” — i.e. the item’s full body carried inside the feed — web.resource.org/…/content (verified 2026-08-21)
    • RFC 4287 (Atom Syndication Format, standards track, December 2005) defines atom:content as the entry’s content element and constrains its cardinality — a ratified standard for the same signal — rfc-editor.org/…/rfc4287.html (verified 2026-08-21)
    • Google documents machine consumption of feeds: “Google accepts RSS 2.0 and Atom 1.0 feeds” as sitemaps, so a named crawler demonstrably parses these documents — developers.google.com/…/build-sitemap (verified 2026-10-18)
    • The RSS 2.0 specification acknowledges the full-content usage explicitly: “An item may also be complete in itself, if so, the description contains the text” — rssboard.org/rss-specification (verified 2026-08-21)

    Limits

    The strongest documented feed consumer reads feeds only as a source of recent URLs. Google’s sitemap documentation describes RSS and Atom as providing “information about recent URLs”, not article bodies. The claimed “index without visiting each page” path is therefore not documented for any named crawler (developers.google.com/…/build-sitemap, verified 2026-10-18). The RSS 2.0 spec blesses excerpt feeds as an equal, conformant choice: “its description is a synopsis of the story, and the link points to the full story”.

    A truncated feed is therefore a publishing decision, not a defect (rssboard.org/rss-specification, verified 2026-08-21). RFC 4287 goes further: “the absence of atom:summary is not an error, and Atom Processors MUST NOT fail to function correctly as a consequence of such an absence” (rfc-editor.org/…/rfc4287.html, verified 2026-08-21). Neither OpenAI’s (developers.openai.com/…/bots) nor Perplexity’s (docs.perplexity.ai/…/perplexity-crawlers) crawler documentation mentions feeds at all (both verified 2026-08-21), and Google states that no additional files or markup are needed for its AI features (developers.google.com/…/ai-optimization-guide, verified 2026-08-21). The audit’s 500-character bar has no source in any specification.

    How it scores

    Informative, weight 0. Every result carries an unconsumed notice. The audit never warns or fails. No feed, or a feed with no items, is not applicable. A feed with items passes, and the share of items with more than 500 characters of content is named in the message.

    Sources