Buttons and links have accessible names
What it checks
AI browser agents identify clickable elements by their accessible name in the accessibility tree. Buttons and links (including icon-only controls) without text, aria-label, or aria-labelledby are invisible to agents, so they cannot navigate the site or trigger actions.
Why it matters
Agents choose which field to fill by matching the user’s intent against the accessible name of a control in the accessibility tree, then write to that control by reference. An input whose name comes from a properly associated <label for>, aria-label or aria-labelledby appears as, for example, textbox “Email address”. An input with only a visual placeholder, or an adjacent unassociated div, appears unnamed. Natural-language lookup then fails, and the agent either skips the field or writes the right value into the wrong control.
Evidence
Form labels and ARIA naming for agent form-filling
The full chain is first-party documented. Anthropic’s find will ‘Search for elements matching a natural-language description such as “search field” or “add to cart button”’, and returns refs in the same tagged format as read_page. form_input then ‘Set a form element’s value directly’ by that ref. The accessible name is literally the lookup key [anthropic-browser-use-tool]. Playwright MCP’s snapshot contents are documented to include ‘form elements (textboxes, checkboxes with accessible names)’. browser_fill_form and browser_type act on snapshot refs [playwright-mcp-snapshots, playwright-mcp-repo]. accname, a W3C Recommendation, fixes the precedence: aria-labelledby over aria-label over the native <label>, alt or title [w3c-accname].
HTML-AAM confirms that ‘label: provides accessible naming for form controls’ [w3c-html-aam]. browser-use keys on the textbox, combobox, checkbox and searchbox roles, plus the a11y properties ‘focusable, editable, settable’ [browser-use-clickable-elements]. Behavioural corroboration: across 300+ trials, agents engaged only where ‘semantic button overlays or off-screen text labels are present’ and ignored purely visual affordances [machine-readable-ads-paper].
Limits
Pixel-based agents (OpenAI computer use, Gemini Computer Use, Anthropic desktop computer use) fill forms from visual layout and do not require programmatic labels at all [openai-computer-use-guide, gemini-computer-use-docs, anthropic-computer-use-tool]. Adoption data shows agents already cope with widespread absence. Only about 35% of mobile inputs get their accessible name from a <label> [web-almanac-2025-accessibility], yet agentic form-filling demonstrably works across the live web. Placeholder text, nearby text and CSS position all provide fallback signal, and Anthropic’s find is explicitly a fuzzy natural-language matcher rather than an exact accessible-name lookup.
No public benchmark isolates label presence against form-fill success rate, so the effect size is undocumented even though the mechanism is airtight. Score presence of an accessible name from any accname-recognised source, not the literal <label for> element.
How it scores
The whole chain is first-party documented. Anthropic’s find locates “elements matching a natural-language description such as ‘search field’ or ‘add to cart button’” and returns refs. form_input then sets a value by that ref. The accessible name is literally the lookup key, so a control named only by a placeholder is not addressable by intent. A named vendor, a named tool and a stated mechanism is grade A. It scopes to programmatic agents: pixel-based ones (OpenAI computer use, Gemini Computer Use, Anthropic desktop computer use) fill forms from visual layout and need no programmatic label at all.
Sources
- Browser use tool (browser_toolset_20260801) — Anthropic, vendor-doc (verified 2026-08-21)
- Snapshots — Playwright MCP — Microsoft, vendor-doc (verified 2026-08-20)
- microsoft/playwright-mcp README — Microsoft, vendor-doc (verified 2026-08-21)
- Accessible Name and Description Computation 1.1 — W3C, spec (verified 2026-08-20)
- HTML Accessibility API Mappings 1.0 — W3C, draft-spec (verified 2026-08-21)
- browser-use ClickableElementDetector source — browser-use, vendor-doc (verified 2026-08-20)
- Machine-Readable Ads: Accessibility and Trust Patterns for AI Web Agents interacting with Online Advertisements — arXiv (Nitu, Mühle, Stöckl, 2025), study (verified 2026-08-20)
- Web Almanac 2025 — Accessibility chapter — HTTP Archive, dataset (verified 2026-08-20)
- Computer use — OpenAI API guide — OpenAI, vendor-doc (verified 2026-08-21)
- Computer use — Gemini API — Google, vendor-doc (verified 2026-08-20)
- Computer use tool — security and prompt injection guidance — Anthropic, vendor-doc (verified 2026-08-20)