catalog: boxIDs for unpainted content (closed <details>, visibility:hidden) don't match the page image — Chromium reports layout boxes for skipped content

[wittjeff]

Summary

The catalog's boxIDs and the in-catalog page image can disagree about elements that are in the DOM but not painted — most commonly content inside closed <details> elements. Chromium returns plausible, non-zero getBoundingClientRect boxes for that content, so the catalog records a boxID that points at a location where the page image shows something else entirely. Anything that draws the catalog box onto the page image (our monitoring UI does) highlights the wrong content.

This is not a regression in the in-catalog shoot — image and boxes do come from the same page load and DOM state. The problem is that getBoundingClientRect alone is no longer a reliable "where is this element visibly" signal in modern Chromium.

Reproduction

https://whatcompublichospital.org/ (chromium, viewport 1280×800, deviceScaleFactor 1). The page has seven <details>; only the first is open. Catalog boxIDs recorded for content inside the closed ones:

element (pathID)	boxID	checkVisibility()
/html/body/main[1]/details[2]/p[1]/a[1]	386:1842:470:25	false
/html/body/main[1]/details[4]/summary[1]	189:1891:902:45	true
/html/body/main[1]/details[4]/p[1]/a[1]	514:1964:348:25	false

Note the interleaving: details[2]'s hidden paragraph boxes sit between details[4]'s summary and details[4]'s hidden content — impossible for any actually-rendered layout. The page image (correctly) shows the collapsed state, so the two checkVisibility() === false boxes overlap unrelated visible content.

Root cause

Chromium renders closed-<details> content as content-visibility: hidden "skipped content" (this is what enables find-in-page into collapsed disclosures / hidden=until-found). Per the CSS containment spec, geometry APIs on skipped content still work — layout is computed on demand — but the content is never painted and following content is not shifted. So the rects are "real" layout values for a rendering that doesn't exist on screen. The same applies to visibility: hidden subtrees. checkVisibility() is the spec-provided discriminator for "is this actually painted."

Proposals (both confined to getCatalog)

1. Visibility guard on boxID — in the catalog page.evaluate, record an empty boxID for elements that fail checkVisibility:

const isVisible = typeof element.checkVisibility === 'function'
  ? element.checkVisibility({checkVisibilityCSS: true, visibilityProperty: true})
  : true;
const domRect = isVisible ? element.getBoundingClientRect() : null;

An empty boxID is already the established "no box" value (same as the current falsy-domRect branch), so consumers degrade gracefully. Browsers without checkVisibility keep today's behavior.

2. Expand disclosures before the shoot and the measurements — immediately after the catalog page loads, before shoot() and before the catalog evaluate:

await page.evaluate(() => {
  document.querySelectorAll('details:not([open])').forEach(details => {
    details.setAttribute('open', '');
  });
});

Rationale: testaro rules routinely flag elements inside collapsed disclosures. With proposal 1 alone they'd simply have no box; with expansion, the page image shows the disclosed content and those elements get accurate, image-aligned boxes. Could be made opt-in (a report field) if you'd rather keep the image strictly as-delivered by default.

No effect on test verdicts: getCatalog runs on its own browser launch in the parent process and that browser is closed before doActs forks the per-tool children, each of which performs its own launch and navigation. The catalog is only consumed via getXPathCatalogIndex (display/location metadata); no test reads catalog geometry. Rules that legitimately test hidden-but-focusable content are unaffected.

We are running both changes in production as a patch and they've been verified against the page above (all sampled boxes land exactly on their elements in the page image; 21 unpainted elements correctly get no box). Happy to submit a PR — together or split per proposal, whichever you prefer.

[jrpool]

Thank you, @wittjeff, for this important discovery. The only disadvantage of proposal 2 that I can think of is that it will enlarge the report. That disadvantage seems to me less significant than the advantage of permitting discovery of offending elements in a screenshot. A separate feature could allow users who have no need for the screenshot to waive it. So, meanwhile, I would be grateful for a PR embodying proposal 2.

[jrpool]

Thanks for these ideas, @wittjeff. I have approved and merged the related PR #71. I accept your proposal to submit a new PR for proposal 1 of this issue.