[BUG] Web | tapOn text+index does not span iframes — silently re-taps top-frame match (#67 fix incomplete)

[richjun]

Description

Follow-up to #67 (closed, marked fixed in v1.1.13 by commits 8ecbbc6 / 4ca8fd8 / ed90309).

tapOn with text + index no longer drops the index field with the old "index" is not supported on web — will be ignored warning — that part of #67 is fixed. But index does not enumerate matches across iframes: when the Nth match would be inside an iframe, the runner silently re-taps the last in-range top-frame match instead, reports tapOn ✓, and never raises an error. The result is a false-positive that's hard to detect — tapOn "succeeded" but the click landed on the wrong button.

Same yaml on upstream Maestro 2.5.1 taps the iframe-hosted match.

Environment

• maestro-runner: 1.1.13 (commit 7addd21, built 2026-05-05T14:07:52Z, go1.26.0)
• OS: macOS 15 (Darwin 25.2.0, arm64)
• Executor: web (Chrome stable, headed via CDP)
• Comparison: Maestro (mobile-dev-inc) 2.5.1 — equivalent flow resolves the iframe-hosted match.

One-line repro

A single self-contained HTML page is hosted on Cloudflare R2 — no app login, no third-party site, no local server. The page has two Help buttons: one in the top frame, one inside an iframe (via srcdoc). Each button's click handler appends a uniquely-labelled marker (TOP_HELP_CLICKED / IFRAME_HELP_CLICKED) to the top frame so the test can assert which one actually received the click.

# repro.yaml
url: "https://pub-4e93ec0f0add49d78a646e058039eb5b.r2.dev/helpidx.html"
name: help-index-iframe-repro
---
- launchApp
- assertVisible: "Help"
- tapOn:
    text: "Help"
    index: 1                              # SHOULD tap the iframe Help (2nd in document order)
- extendedWaitUntil:
    visible: "IFRAME_HELP_CLICKED"        # FAILS — iframe button never received the click
    timeout: 3000

maestro-runner --platform web test repro.yaml

Result matrix

Three cases were run against the same R2-hosted page on maestro-runner 1.1.13:

#	tapOn	post-tap assertion	maestro result	what actually happened
A	text: "Help", index: 0	extendedWaitUntil visible "TOP_HELP_CLICKED"	✓ PASS	top-frame Help was tapped — correct
B	text: "Help", index: 1	extendedWaitUntil visible "IFRAME_HELP_CLICKED"	✗ FAIL (3s timeout)	iframe Help was not tapped
C	text: "Help", index: 1	extendedWaitUntil visible "TOP_HELP_CLICKED"	✓ PASS	top-frame Help was tapped again (the same one as case A)

Case C is the smoking gun: with only one top-frame Help button, index: 1 should resolve to the iframe-hosted Help (or fail with not-found). Instead it silently re-resolves to the top-frame Help — same element as index: 0. Maestro reports tapOn ✓ either way.

Verbatim run output

Case A (index: 0):

✓ launchApp                                            (527ms)
✓ assertVisible: text="Help"                           (5ms)
✓ tapOn: text="Help"                                   (54ms)
✓ extendedWaitUntil: visible text="TOP_HELP_CLICKED"   (0ms)
help-index-0-r2    ✓ PASS    4 steps    4 pass    0 fail    0 skip    595ms

Case B (index: 1 + assert iframe marker):

✓ launchApp                                                (530ms)
✓ assertVisible: text="Help"                               (3ms)
✓ tapOn: text="Help"                                       (44ms)   ← reports success
✗ extendedWaitUntil: visible text="IFRAME_HELP_CLICKED"    (3.0s)   ← but iframe button was never clicked
  ╰─ Wait condition not met within 3s
help-index-1-r2    ✗ FAIL    4 steps    3 pass    1 fail    0 skip    3.8s

Case C (index: 1 + assert top marker — proves the silent fall-back):

✓ launchApp                                            (528ms)
✓ assertVisible: text="Help"                           (4ms)
✓ tapOn: text="Help"                                   (43ms)
✓ extendedWaitUntil: visible text="TOP_HELP_CLICKED"   (1ms)        ← top button hit again
help-index-1-r2-checktop    ✓ PASS    5 steps    5 pass    0 fail    0 skip    742ms

A screenshot taken in Case C confirms only TOP_HELP_CLICKED is rendered after the index: 1 tap — IFRAME_HELP_CLICKED is absent and the iframe Help button is visibly untouched.

Repro page source — helpidx.html (single file, ~1.8 KB)

The hosted page is the self-contained file below. Copy into helpidx.html and serve from anywhere if you'd rather not hit R2.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>maestro-runner Help text+index test</title>
  <style>
    body { font-family: sans-serif; margin: 24px; background: #fff; color: #222; }
    button { padding: 8px 16px; margin: 4px; cursor: pointer; }
    iframe { width: 600px; height: 220px; border: 1px solid #999; display: block; margin-top: 12px; }
    .marker {
      margin-top: 16px; padding: 8px 12px;
      background: #fff3cd; border: 1px solid #ffeeba;
      border-radius: 4px; display: inline-block;
      font-weight: bold;
    }
  </style>
</head>
<body>
  <h1>Help index test</h1>
  <p>
    Top-frame <code>Help</code> button is the first occurrence in document order.
    The iframe below (via <code>srcdoc</code>) contains a second <code>Help</code> button.
    Whichever button receives the click appends a uniquely-labelled
    <code>&lt;div class="marker"&gt;…&lt;/div&gt;</code> to the top frame so the test can assert
    which one was actually hit.
  </p>

  <button id="top-help">Help</button>

  <iframe id="inner" title="inner iframe with second Help" srcdoc='
<!DOCTYPE html>
<html lang="en">
<body style="font-family:sans-serif;margin:16px">
  <button id="iframe-help">Help</button>
  <p>Help button inside iframe (second occurrence in document order).</p>
  <script>
    document.getElementById("iframe-help").addEventListener("click", () => {
      parent.document.body.insertAdjacentHTML(
        "beforeend",
        "<div class=\"marker\">IFRAME_HELP_CLICKED</div>"
      );
    });
  </script>
</body>
</html>
  '></iframe>

  <script>
    document.getElementById("top-help").addEventListener("click", () => {
      document.body.insertAdjacentHTML(
        "beforeend",
        "<div class=\"marker\">TOP_HELP_CLICKED</div>"
      );
    });
  </script>
</body>
</html>

Expected

tapOn { text: "Help", index: N } enumerates Help matches in document order across the top frame and every same-origin iframe (consistent with how 8b33eb2 already walks iframes for "find one match"), and taps the Nth match. If N is out of range, surface a clear not found error rather than silently re-tapping a top-frame element. This matches upstream Maestro 2.5.1.

Likely root cause

The index selector resolution path in pkg/driver/browser/cdp/finder.go (added in 8ecbbc6 / 4ca8fd8) appears to enumerate matches only within the top frame when text is combined with index. The same-origin iframe walk added in 8b33eb2 (_collectDocs() / _findMatchingElements) is not threaded through the text+index code path, so iframe matches are never visible to the index slot. When index >= len(top-frame matches), the resolver clamps or falls back to the last in-range top-frame match instead of erroring.

Suggested fix

In the text + index branch, build the candidate list from the same _collectDocs()-driven scan that the rest of the iframe-aware finders use, sort by document-order across (top, iframe-1, iframe-2, …), index into the unified list, and on out-of-range raise a text 'X' index N: only K match(es) found error rather than tapping the closest in-range element. Add a peer test to pkg/driver/browser/cdp/finder_iframe_test.go covering the index: 1 → iframe-hosted match case.

Happy to open a PR.

[omnarayan]

@richjun Thanks for the precise diagnosis

tapOn text+index across iframes

The "silent re-tap of the closest in-range top-frame match" framing in your bug nails the worst part of this — a green tapOn ✓ while the actual click went somewhere meaningless. The result-matrix table you put together (Cases A/B/C) was the cleanest demonstration I've seen of the issue, and I ended up using it verbatim as the verification suite for the patch.

What's fixed

The text + index resolver now enumerates matches across every same-origin root reachable from the top frame — top document, same-origin iframe documents, and (on top of #71) open shadow roots. Out-of-range no longer falls back to a top-frame element; it surfaces a clear error with the actual count.

A couple of implementation notes worth calling out:

1. Strict semantic matching. Earlier in the patch I was matching on textContent substring/transitive match, which inflated the candidate list with every <p>, <h1>, <code> ancestor that happened to contain "Help" in its textContent — your repro page produced 9 such matches. With strict own-text plus an interactive-element filter (<button> / <a> / <input> / <select> / <textarea> / [role] / [tabindex] / [aria-label]), the candidate list comes back as the 4 elements you'd actually mean. Approximates Maestro CLI's AX-tree-named-element behaviour without the per-frame Accessibility.queryAXTree round-trip cost.

2. Two-stage call to dodge a Rod stall. First eval returns the count by value; second eval (only when in range) returns the element handle via .ByObject(). Hit a real stall during development where rod.Eval(...).ByObject() resolving against a JS null hung the eval pipe — no remote objectId to track. Same stall pattern Playwright eliminated structurally years ago by force-rejecting all pending CDP callbacks on session close. Two-stage avoids the situation entirely.

Verification against your three cases

Case	Setup	Pre-fix	Post-fix
A	index: 0, assert TOP_HELP_CLICKED	✓ control	✓ pass
B	index: 1, finding	✗ silently re-tapped top → TOP_HELP_CLICKED	✓ resolves to iframe <button id="iframe-help">
C	index: 5 (out of range)	✓ silent re-tap of top button (false-positive)	✗ tapOn fails with text 'Help' index 5: only 2 match(es) found across same-origin roots, after the configured timeout

Caveat — Case B's downstream assertion

Your Case B asserts IFRAME_HELP_CLICKED after the tap. With this fix the runner finds the iframe <button> correctly and dispatches a click — but currently at iframe-local viewport coordinates instead of top-frame coords, so the click misses and the iframe handler doesn't fire. Same iframe coordinate-translation bug that bites #71's tapOn: "Close". Both are tracked together in [#NN — to be filed]; the fix ports Playwright's _checkFrameIsHitTarget walk plus post-click elementFromPoint verification (closed Playwright #13762).

So 1.1.14 fixes the finding layer — no more silent re-taps, accurate index, clear out-of-range errors. The acting layer through iframe boundaries needs the follow-up. Honest framing: your Case B's tapOn will return success (because it dispatched), but the assertion on IFRAME_HELP_CLICKED will time out until the coord-walk lands.

[omnarayan]

Fixed in v1.1.14. tapOn with text + index now walks every same-origin root via _collectRoots(), sorts by document order, and indexes deterministically — out-of-range returns a precise error with the actual match count instead of silently re-tapping the closest in-range top-frame match. Also tracked in the dispatch-layer follow-ups (PR #73, PR #74) which fixed the coord-translation and Flutter Web hit-target verification that this issue's repro indirectly surfaced. See commit 410c3f4. Thanks for the three-case matrix repro — the case-C smoking gun was exactly what we needed.