Opens in a new tabSkip to content
Agent LighthouseAgent Lighthouse

    Searches the text of every published page. The evidence sources themselves are not in this index — search all of them on the trusted sources page.

    GitHub ↗
    Browse checks and page contents
    answer-readiness/section-split-risk-profile

    Section split-risk profile

    What it checks

    Measures every h2 and h3 section against the published default chunk window of 512 tokens, or about 2000 characters. It finds sections that will be mechanically cut into two or more chunks, producing tail chunks that carry no heading. It also finds the inverse: sections too thin to embed meaningfully. Also flags atomic structures (tables, long ordered lists) longer than the window, which get split mid-structure.

    Why it matters

    Fixed-window splitters cut at a character/token budget, not at meaning. When a section exceeds the window, chunk 1 keeps the heading (the strongest query-matching signal on the page) and every subsequent chunk from that section is headless — its embedding loses the topical anchor. Azure publishes 512 tokens / 2000 chars with 25% overlap as the recommended default and lists heading-based segmentation as an alternative that avoids this. Falsifiable prediction: for a page with one 2,000-token section versus the same content split into four 500-token h2 sections, the queries that match content in the final quarter of the text retrieve the split version and miss the monolithic one.

    Evidence

    • Chunk Documents - Azure AI Search — Microsoft (vendor-doc, verified 2026-10-17)
    • Recommends “starting with a chunk size of 512 tokens (approximately 2,000 characters) and an initial overlap of 25%”. This is the window the audit measures against.
    • Its default Text Split parameters cut pages of 2,000 characters with a 500-character overlap.
    • It names heading syntax in HTML or Markdown as a way to chunk by section. It also suggests appending the document title to mid-document chunks “to prevent context loss”, which is the loss a headless tail chunk suffers.
    • Split HTML — LangChain text splitter integrations — LangChain (vendor-doc, URL verified 2026-08-20)
    • HTMLHeaderTextSplitter splits on <h1>, <h2>, <h3> and attaches the headers relevant to each chunk as metadata. A section with headings carries them into its chunks. A page without them gives the splitter nothing to cut on.

    Grade: B — Microsoft documents the 512-token default for its own retrieval pipeline, and LangChain documents heading-based splitting. Neither measures the retrieval loss from a headless tail chunk, and other pipelines use other windows. A documented default with no measured effect is a grade-B mechanism.

    How it scores

    Tier per evidence policy: scored — grade B meets the A/B bar required for scored audits.

    Example finding

    A 3,100-token ‘Frequently asked questions’ h2 with 22 questions as bold paragraphs rather than h3 headings. Split at 512 tokens it yields six chunks, five of which are headless bare Q&A text with no page or topic anchor. Promoting each question to h3 converts the same content into 22 self-anchored chunks.

    Sources