We assess the proof before a check can affect your score. Documented support matters. A popular idea alone is not enough.

Advisory and experimental checks do not change the score. The policy below explains the grades, limits and review rules.

Evidence Policy

Agent Lighthouse scores websites on their readiness for AI agents. A wrong audit is worse than no audit: it sends site owners to do work that has no effect, and it erodes trust in every other result. This policy defines what counts as proof, how audits are graded, and what each grade is allowed to do to a score.

The core rule

No audit contributes to a score unless its mechanism is proven at grade A or B.

Every audit must state its mechanism as a falsifiable causal claim — “GPTBot fetches /llms.txt before crawling”, not “AI systems prefer well-structured sites”. Vague claims cannot be proven or refuted, so they cannot be scored.

A grade licenses the mechanism it was proven for, and nothing wider. If the mechanism is about an artifact’s contents — an OpenAPI document’s servers array, a feed’s lastmod dates — the grade covers sites that publish that artifact. It says nothing about a site that publishes none, because no vendor documents a consumer for the absence. An audit about an artifact’s contents therefore returns notApplicable when the artifact is absent; only a present-and-defective artifact may fail. The grade-C row below records the same reflex from the other side: Chrome Lighthouse scores a missing llms.txt as notApplicable.

Evidence grades

What grades are allowed to do

Informative or obsolete

Both tiers carry weight 0. They make different claims, so they need different sources.

• Informative: the thing is live. No consumer documents reading it yet. The audit may grade it, and may carry a notice of kind unconsumed (“no AI vendor reads this”).
• Obsolete: a consumer withdrew the thing, a vendor documents ignoring it, its spec is dead, or the governing project documents a successor in its place (superseded: the project considered the old location and chose another). The audit carries grade D, because the grade states the strongest proven consumer path and there is none. Its notice, of kind obsolete, cites the dated source for the retirement and names the replacement where one exists.

An obsolete audit answers one question: does this site still publish the retired thing? Absent is notApplicable. Present is warn. It never returns fail, because a dead file harms nothing; it only wastes the owner’s effort. A “no consumer” finding alone is not enough for the obsolete tier. Calling a live but unread thing “retired” claims more than any source documents.

Source requirements

• Every claim cites a source id from sources.json — one registry, no scattered links.
• Primary sources outrank secondary: vendor docs and spec text over blog posts over social media.
• Every source URL is verified to resolve at research time; the registry records the access date.
• Counter-evidence is recorded, not hidden. If Google states it ignores a file, the dossier for that file’s audit says so. Credibility comes from showing the evidence against our own checks.
• CI runs a link checker over the registry. A dead link fails the build; a source older than the citing audit’s review window triggers a re-review flag.

Re-review cadence

The AI-agent ecosystem changes quarterly. Every audit’s dossier carries a reviewed: date. An audit whose evidence is older than its grade’s review window (A: 12 months; B: 6 months; C and D: 3 months), or whose cited standard changes status (ratified, deprecated, abandoned), is re-graded. Downgrades below B remove the audit from scoring in the next minor release.

Deprecation process

Audits die publicly, not silently.

When owners may still publish the thing the audit finds, the audit moves to the obsolete tier instead of leaving. The report keeps telling them it is retired, which is more use to them than silence. The move is a major release, because the tier and grade change.

When there is nothing to tell an owner (the audit judged a practice, not an artifact, and its evidence fell away):

1. The dossier’s status changes to deprecated, with the evidence for removal cited.
2. The audit ships one more minor release as informative (weight 0) with a notice of kind deprecated in the report.
3. The next major release removes it. The dossier stays published as the record of why.