Skip to content

fix(runner): read a blind fill-point lookup as unreadable, not empty - #3972

Open
thymikee wants to merge 2 commits into
mainfrom
maint/1721-blind-fill-point-lookup
Open

thymikee wants to merge 2 commits into
mainfrom
maint/1721-blind-fill-point-lookup

Conversation

@thymikee

@thymikee thymikee commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Summary

A degraded iOS XCTest channel can return an empty app tree without an issue. The shared text-input probe reports that read as unavailable; a readable screen with no matching input reports NO_TEXT_INPUT_AT_POINT. Coordinate, ref, selector, and replay fills refuse before dispatch when the lookup cannot answer. Bound-input checks also avoid treating a blind probe as proof that an input is gone.

Four files: runner probe, test flag, simulator regressions, and command documentation. Scope extends the original point lookup to all shared probe callers. Refs #1721.

Validation

Head 55746e00accb9caadb3831c89f9b5afea94935e2; original base 5b61e8a9ad3d16d34c4462ca6749937571b54c88.

  • pnpm check:affected --run --head 55746e00ac: passed (exactCommit=true, checkoutChanged=false); five runnable checks passed. Device and replay lanes were skipped locally.
  • Apple smoke attempt 2 passed, including smoke:form-input. Attempt 1 failed with reordered text; cause unproven. All 20 PR checks passed at head, including Coverage and Integration Tests.
  • Both requested simulator regressions passed in the full XCTest suite. That suite failed two in-app-back navigation tests; their cause is unproven. See the test results comment.
  • The blind-tree case injects the read outcome; no real degraded channel or latency comparison was obtained. Cold self-review only; no independent reviewer was available.

A degraded XCTest channel is known to enumerate nothing on some
simulator trees while gestures still land, and that reads as a clean
empty query. An enumeration that saw no element cannot support the
claim that the point holds no input, so the pre-tap lookup answers
unavailable unless it enumerated something to filter.

Refs #1721
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-3972/

Built to branch gh-pages at 2026-10-26 13:33 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.54 MB 5.54 MB +22 B
Package (unpacked) 5.54 MB 5.54 MB +22 B
Package (download) 1.68 MB 1.68 MB +27 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 36.0 ms 34.5 ms -1.5 ms
CLI --help 68.5 ms 69.3 ms +0.8 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread website/docs/docs/commands.md Outdated
@thymikee

Copy link
Copy Markdown
Member Author

Thanks for this. At da50c61 one problem remains: a fill aimed at a screen with no text input now fails as TEXT_INPUT_PROBE_UNAVAILABLE instead of NO_TEXT_INPUT_AT_POINT. CI is green (19 checks, none failing), and there are no conflicts.

guard !enumerated.isEmpty else { return .unavailable } treats an empty result as a blind channel. The enumeration only queries textFields, secureTextFields, searchFields and textViews. A healthy, fully readable screen with no text input returns the same empty list, for example a ref, selector or coordinate fill aimed at a button or label on the degraded route. That true "absent" used to answer NO_TEXT_INPUT_AT_POINT. It now answers TEXT_INPUT_PROBE_UNAVAILABLE, and its hint says to retry a fill that cannot succeed. The fill still refuses before the tap and no field changes, but callers branch on the reason code, so the change swaps a false "absent" for a false "unreadable". The existing absent test uses a screen that has a field, and the new test empties the queries on a screen that has a field, so nothing covers a screen with no input. The rule should be: the probe is unavailable only when the app tree itself reads as empty or root-only (the condition snapshot.ts:331 names), and absent when the tree reads something but no input matches the point. An app-wide read taken inside the same issue containment would give that signal, for example the first element of app.descendants(matching: .any) or the window count. Please keep the typed unavailable reason, and add a regression next to the blind test that launches the fixture with no text input and expects NO_TEXT_INPUT_AT_POINT.

Could the "tree read nothing" signal be evaluated once in the probe owner, and returned as unavailable for every caller of the enumeration? probeTextInputs already returns TextInputProbeOutcome, and it still returns .absent for the same empty enumeration at RunnerTests+TextEntry.swift:541 and :576, where boundTextEntryInputIsGone reads it as "input proven gone". Moving the rule there, with a new TextInputProbeFailure case, and having coordinateTapTextInputIdentityAt reuse probeTextInputs, would keep about the same line count, remove the queryTextInputs split and fix the issue above. The first step is to decide the blindness signal and keep it in the probe owner.

Not blocking, and fine to take or leave: after the fix, re-check the two bullets in commands.md, because "the runner read the screen" and "reading no elements at all" over-claim when an empty result only means no text input was found.

The open inline thread on the TEXT_INPUT_PROBE_UNAVAILABLE bullet scope (#3972 (comment)) still applies as a one-clause doc fix, since both routes reach the one point route.

I did not run Swift or XCTest. The base-red, head-green and mutation results are the author's runs, and I only reasoned from the diff that the new test fails without the guard. The packet does not show that the ios.yml runner lane executed the new test at this commit. No live run exists for the new unreadable state, since the degraded channel cannot be produced on demand, so the fixture flag is the only evidence for it.

@vercel

vercel Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
agent-device-website Ignored Ignored Preview Oct 29, 2026 12:55pm UTC

Request Review

@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the update. At 55746e0 the code now handles the earlier no-input screen finding and the docs now match the behavior, but the new check has not been run yet.

The probe now does a new live read, app.descendants(matching: .any).firstMatch.exists, in RunnerTests+TextInputProbe.swift. Every coordinate, ref, selector and replay fill on a penalized channel goes through it. The PR body says the two new XCTest cases were not run at this head. Static reading cannot tell us if the read is true on a healthy screen with zero inputs, which is the exact regression raised before. It also cannot tell us if the read is false on a real degraded tree. Please run both tests on an iOS simulator at this head and paste the passing xcodebuild output into the PR. testSynthesizedReplacementRefusesABlindEnumerationAsUnreadableNotEmpty must show TEXT_INPUT_PROBE_UNAVAILABLE. testSynthesizedReplacementOnScreenWithoutInputsReportsAbsent must show NO_TEXT_INPUT_AT_POINT.

The Smoke Tests job failed in smoke:form-input (run 40877379186). fill id=field-email left "ada@example.estt" with no runner refusal code. This looks like the known typing-pace character reorder (#2080), not this diff, because a refusal from the probe would have shown one of the two codes above. The log does not show if the channel was penalized, so I cannot rule out overlap. Please rerun the job.

Not blocking, and you can take or leave these: the blind-tree test seam overwrites treeReadable after the real .exists read, so the test proves the branch below it and not the read itself, which you already disclose. The fallback .unavailable(.emptyTree) at the same read is always replaced by .failure(.exception) when used, so a neutral fallback such as .unavailable(.exception) would be clearer. Each probe also adds one full-tree query before the four enumeration queries, so keep it only if an executed run shows no latency regression.

The resolved cubic-dev-ai thread on the docs wording is fixed: both bullets and the reason list in commands.md now name the coordinate, ref, selector and replay fills. #3972 (comment)

I see no code issue beyond the missing run. Before merge, the two XCTest cases must pass at 55746e0 with the output in the PR, and Smoke Tests must pass on a rerun.

@thymikee

Copy link
Copy Markdown
Member Author

At 55746e00accb9caadb3831c89f9b5afea94935e2, the iOS simulator XCTest run produced:

Test Case '-[AgentDeviceRunnerUITests.RunnerTests testSynthesizedReplacementOnScreenWithoutInputsReportsAbsent]' passed (5.045 seconds).
Test Case '-[AgentDeviceRunnerUITests.RunnerTests testSynthesizedReplacementRefusesABlindEnumerationAsUnreadableNotEmpty]' passed (7.695 seconds).

The first asserts NO_TEXT_INPUT_AT_POINT; the second asserts TEXT_INPUT_PROBE_UNAVAILABLE with an injected blind-tree read. The run did not reproduce a real degraded tree or measure the added query's latency.

The full suite failed two other cases: testInAppBackStillAsksForWindowsWhereNoSoleScreenAnswers and testInAppBackVerifiesSoleScreenWhenXCTestCannotReadFramesOrEnumerate. Both reported an unchanged screen after the synthesized back gesture. Their cause has not been established; the suite is not a pass.

Apple smoke attempt 2 passed, including smoke:form-input. The cause of attempt 1's reordered email remains unproven. All 20 PR checks passed at this head.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant