Summary
Empirical measurement of the hackbrowser crawling engine against a battery of modern single-page-app interaction patterns: drawer/menu navigation, a searchable (MUI-style) combobox, animated overlays, element ids that regenerate on re-render, auto-closing transient UI, and widgets that mount on a timer. All findings are from synthetic/localhost repro — no external target involved.
The headline result: most of these patterns are already handled today. Two reproducible gaps remain, and both share one root cause — time-dependent DOM: the crawler plans against a DOM snapshot that is no longer valid by the time it acts.
Handled (verified, no action)
- SPA button-navigation via
history.pushState (nav items are <button onClick=navigate()>, no <a href>, no domcontentloaded refire) — discovered and followed; content behind the route is captured.
- Searchable combobox whose options mount ~600 ms after it opens — completed by the mechanical combobox probe path (type → wait → select).
- Element ids that regenerate on every re-render — re-resolved before the action.
- Animated backdrop / overlay occlusion — dismissed by the occlusion-recovery path.
- Input + submit button not wrapped in a
<form> — paired and submitted.
- Auto-closing drawer, in isolation under a fast planner — the revealed item is reached before the close fires.
Gaps
A. Transient UI decays before the crawler acts — high
A container that must be opened to reveal its actions (drawer, menu, popover) and that decays on a wall-clock timer (auto-close after N seconds) closes before the crawler clicks the revealed item.
Root cause: the crawler opens the container, then makes an LLM planning round-trip to decide the follow-up click. That gap (a few seconds under a fast planner, much larger under a slower one) exceeds the decay timeout, so the revealed items go off-screen / detached and every click times out.
Isolated by additive bisection: with several other stressors present but the auto-close timeout raised, the crawl succeeds; with the timeout alone shortened, it fails. The decay-vs-planning-latency race is the sole determinant — animation, ephemeral ids, and backdrop occlusion only shift the margin, they are not the cause.
- Repro: a localhost drawer fixture — open (auto-close 1.5–3 s) → item. Items are discovered, but all clicks time out because the drawer has already closed.
- Fix direction: treat open-then-reveal triggers mechanically, the way the combobox path already works — after opening a container, collect and act on the revealed items without a full planning round-trip, shrinking the open→act gap below typical decay timeouts. And/or add a re-open-and-retry recovery when a revealed item is detached/off-screen at click time.
B. Content mounted via a timer after load is missed — low
An element injected purely via setTimeout (no network activity) after the scan runs is never seen: the render-wait guard fires only on a zero-element scan, and the network-idle stabilization does not catch timer-driven mounts (there is no in-flight request to wait on).
- Repro: a localhost page where a button is appended ~4.5 s after load; the crawl scans early, plans nothing, and never re-scans.
- Common source: third-party widgets (chat / support) — frequently out of scope, hence low priority.
- Fix direction: a DOM-stable (mutation-quiescence) wait or a bounded periodic re-scan, gated to avoid cost on persistent-connection pages.
Common theme
Both gaps are the same assumption failing: the crawler treats the DOM it planned against as still valid when it acts. Time-dependent UI — content that decays (A) or appears late (B) — breaks that assumption. The fix theme is symmetric: shrink or re-validate the plan→act window against a moving DOM.
Methodology note
Measured by isolating each pattern in a single-trap localhost fixture (mechanism copied faithfully from the in-repo mui-spa-trap test fixture), then additively combining to find the minimal breaking set. Two fixture confounds were caught and corrected mid-measurement (a lone-textbox variant that changed the combobox's character; a self-occluding closed drawer) — "each pattern individually handled" did not imply the combination works, which is how gap A surfaced.
Summary
Empirical measurement of the hackbrowser crawling engine against a battery of modern single-page-app interaction patterns: drawer/menu navigation, a searchable (MUI-style) combobox, animated overlays, element ids that regenerate on re-render, auto-closing transient UI, and widgets that mount on a timer. All findings are from synthetic/localhost repro — no external target involved.
The headline result: most of these patterns are already handled today. Two reproducible gaps remain, and both share one root cause — time-dependent DOM: the crawler plans against a DOM snapshot that is no longer valid by the time it acts.
Handled (verified, no action)
history.pushState(nav items are<button onClick=navigate()>, no<a href>, nodomcontentloadedrefire) — discovered and followed; content behind the route is captured.<form>— paired and submitted.Gaps
A. Transient UI decays before the crawler acts — high
A container that must be opened to reveal its actions (drawer, menu, popover) and that decays on a wall-clock timer (auto-close after N seconds) closes before the crawler clicks the revealed item.
Root cause: the crawler opens the container, then makes an LLM planning round-trip to decide the follow-up click. That gap (a few seconds under a fast planner, much larger under a slower one) exceeds the decay timeout, so the revealed items go off-screen / detached and every click times out.
Isolated by additive bisection: with several other stressors present but the auto-close timeout raised, the crawl succeeds; with the timeout alone shortened, it fails. The decay-vs-planning-latency race is the sole determinant — animation, ephemeral ids, and backdrop occlusion only shift the margin, they are not the cause.
B. Content mounted via a timer after load is missed — low
An element injected purely via
setTimeout(no network activity) after the scan runs is never seen: the render-wait guard fires only on a zero-element scan, and the network-idle stabilization does not catch timer-driven mounts (there is no in-flight request to wait on).Common theme
Both gaps are the same assumption failing: the crawler treats the DOM it planned against as still valid when it acts. Time-dependent UI — content that decays (A) or appears late (B) — breaks that assumption. The fix theme is symmetric: shrink or re-validate the plan→act window against a moving DOM.
Methodology note
Measured by isolating each pattern in a single-trap localhost fixture (mechanism copied faithfully from the in-repo
mui-spa-traptest fixture), then additively combining to find the minimal breaking set. Two fixture confounds were caught and corrected mid-measurement (a lone-textbox variant that changed the combobox's character; a self-occluding closed drawer) — "each pattern individually handled" did not imply the combination works, which is how gap A surfaced.