Skip to content

fix(apple-runner): serve a macOS read of a background app without raising it - #3339

Merged
thymikee merged 5 commits into
mainfrom
fix/macos-background-activation-3254
Oct 9, 2026
Merged

thymikee merged 5 commits into
mainfrom
fix/macos-background-activation-3254

Conversation

@thymikee

@thymikee thymikee commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Summary

On the default XCTest backend, a macOS read (snapshot, get, find, an interaction's leading reads) of a session app sitting behind other windows raised it first, taking the user's frontmost app away for a command that only asked to look (#3254). The fix rides the existing launchPolicy axis: the .existingApp arm of prepareActiveCommandContext refreshes cached target identity through the shared rule, then serves a .runningBackground app in place via resolveAppWithoutActivation, booking no activation fact. Scope kept exact: stopped apps keep launching (notRunningRefusal stays iOS-only), interactions keep activating, and window-level screenshot keeps its measured-load-bearing raise while --fullscreen of a running app no longer raises or pays its 0.5 s settle. Only .runningBackground is served in place; every other state keeps the activating route, matching targetNeedsActivation's macOS set.

Background reads answer as ordinary reads — decided, not omitted; XCTAssertNil(prepared.observation) plus the per-state table own the silence. Decision record: #3338.

Validation

Head e0bd78428: check:affected --run passes; host lane 295/295, 0 failures (selection: host lane reaches 295). Canary: removing the arm fails the read test with a bundle_changed fact; removing the identity refresh leaves the planted dead pid. Live run (runner.log attached in review reply): reads answered with Finder frontmost and zero ACTIVATE_FACT lines; the tap after them booked priorState=3; fullscreen raised nothing. CI owns Coverage/replay-macos.

Still activates (XCTest)

Interactions; open/activate/home/recordStart; mouseClick; window-level screenshot of a non-foreground app; any screenshot of a stopped app.

Dispositions

Proposals 1/2/4 already exist on native/accessibility/SCContentFilter paths; proposal 3's overload is argued against in #3338.

Partial #3254

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-09 18:22 UTC

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.13 MB 5.13 MB +1.2 kB
Package (unpacked) 5.13 MB 5.13 MB +1.2 kB
Package (download) 1.54 MB 1.54 MB +301 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.8 ms 27.3 ms +0.5 ms
CLI --help 81.6 ms 82.7 ms +1.1 ms

@thymikee
thymikee force-pushed the fix/macos-background-activation-3254 branch from dd12a41 to 8ffc692 Compare October 8, 2026 20:48

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/commands/schema/cli-help.ts Outdated
…sing it

On a desktop, the XCTest foreground repair is not a no-op for the user: a read that
names a session app sitting behind other windows took their frontmost app away for a
command that only asked to look (#3254). The activation axis already routes by
`launchPolicy`, so the fix goes on the existing `.existingApp` arm of
`prepareActiveCommandContext`: on macOS, a read whose app answers
`.runningBackground` is served in place through `resolveAppWithoutActivation`, and
no activation fact is booked for it. Scoping is exact:

- a stopped app keeps the launch the activating route performs for it today: `state`
  answers `.notRunning` and the arm declines, so `open`-then-read still works and no
  macOS refusal is invented (`notRunningRefusal` stays iOS-only);
- interactions keep activating: the macOS XCTest path drives events through the
  foreground window, and a background click would land on whatever window is on top;
- a window-level screenshot keeps its raise (measured: a backgrounded
  `screenshotRoot(app:).screenshot()` returns the occluding window's content inside
  the session window's rect), but a `--fullscreen` capture of a running app no longer
  raises — `XCUIScreen.main` answers from any foreground, and the 0.5 s settle waits
  belong to the raise and are skipped with it. A stopped app keeps launching either way.

The two conditions are functions the call sites read (`macReadMayBeServedInBackground`,
`macAppCaptureNeedsRaise`) so the host lane pins every state they branch on. Four
host-lane tests: the background read served without a raise or a fact across its three
shapes, the interaction on the same app still activating, a stopped read still
launching, and the raise-condition table. The canary is the arm's removal: the read
test goes red with the app left foreground and a `bundle_changed` fact booked. A live
run on the default backend confirms it: with Finder frontmost, `snapshot` and `get`
answered from the background app, Finder stayed frontmost, and runner.log booked zero
activation facts; the click after them activated with priorState=3 as it must.
`help macos` states the new read behavior.

Also records at the `.presentedSurface` arm why its macOS route is unreachable from
the host (`alert` answers through the helper; `action-button` is refused by its owner
fact before dispatch), so no second activation axis is proposed against it.

Partial #3254
@thymikee
thymikee force-pushed the fix/macos-background-activation-3254 branch from 83304bd to 2df8924 Compare October 8, 2026 21:45
@thymikee thymikee changed the title fix(apple-runner): refuse a macOS read of a stopped app instead of launching it fix(apple-runner): serve a macOS read of a background app without raising it Oct 8, 2026
Turns two accidents into rules, both asked for in coordinator review of #3339:

- `macReadMayBeServedInBackground` splits its state probe from the `state` call so the
  host lane can pin every state the answer branches on. The pin records why only
  `.runningBackground` is served in place: on the macOS SDK the suspended case does not
  exist (the split is pinned by RunnerTests+ApplicationStateRawValueTests), and every
  other state keeps the activating route so an app that cannot promise an answerable
  tree pays the repair rather than trading a focus steal for an empty read — the same
  set `targetNeedsActivation` uses on macOS, so read and capture cannot disagree.
- The disclosure decision is made with the reviewer rather than by omission, and pinned:
  a background-served read books no activation fact and no substitute marker. The
  existing channels cannot say it honestly — the activation fact records only a repair
  performed, and the observation payload belongs to the iOS observe-only contract, which
  refuses exactly this situation. A background tree is live, not degraded (measured
  rect drift is time, not foreground state), and interactions still activate, so no
  action path consumes a background tree. `XCTAssertNil(prepared.observation)` makes the
  absence owned: a future disclosure must edit the assertion on purpose.

No behavior change; both live-state tests and the new table pass on the host lane.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

The raise table claimed to pin every state `macAppCaptureNeedsRaise` branches on but
omitted `.unknown`, while the read table in the same test pinned it — the state worth
pinning on one side could not be unpinnable on the other. The rows convert the silent
drift path into a red row: `.unknown` raises for a window-level capture, where the
activation is the standing route's attempt to settle an app whose state the SDK cannot
report, and stays false for `--fullscreen`, whose pixels answer from any foreground.
Expectations verified against the implemented condition, not pasted. No behavior
change.

Partial #3254
@thymikee

thymikee commented Oct 9, 2026

Copy link
Copy Markdown
Member Author

This PR is ready for review at 74f3899. The Swift runner change and the help string look correct, and all 19 checks pass, including the macOS runner lanes. No conflicts.

Not blocking, and you can take or leave these: (1) in the macOS background arm at https://github.com/callstack/agent-device/blob/74f3899/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift#L552, a read served in place should use the same target identity a bound read would use, so please call refreshCachedTargetIfProcessChanged(bundleId:) before resolveAppWithoutActivation (it refreshes without activating), or explain why a stale instance cannot occur on macOS; I could not confirm whether a cached app instance keeps addressing a dead pid after an outside relaunch, so is the stale pid case possible here? (2) Several comments carry decision history, such as the doc on macReadMayBeServedInBackground (https://github.com/callstack/agent-device/blob/74f3899/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift#L789) and the screenshot comment in CommandExecution.swift; please cut each to the one invariant it protects and move the decision record to #3338 or an ADR. (3) The @MainActor overload at https://github.com/callstack/agent-device/blob/74f3899/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift#L813 builds a fresh app state for every macOS .existingApp read and prepareActivatedTarget queries it again, so reading the state once and passing it down would avoid the second query.

The live run is reported by the author only, and I did not see the runner.log lines with the ACTIVATE_FACT markers, so please attach them. I also did not verify whether element.isHittable at Interaction.swift:280 reports false for elements in an occluded background window on macOS. The runner unit tests were not run on my side.

The fixes for the three resolved inline threads are in at this commit: the removed APP_NOT_RUNNING sentence in cli-help.ts (#3339 (comment)), the region-grab claim now limited to window-level captures (#3339 (comment)), and the LifecycleTests raise rows that now match macAppCaptureNeedsRaise (#3339 (comment)).

A read served in place resolved through `resolveAppWithoutActivation`, which returns the
cached handle when the cache names the app — and a cached handle can record a pid an
outside relaunch replaced. The activating route never answers from that residue because
it runs `refreshCachedTargetIfProcessChanged` before its `targetNeedsActivation` read;
the new arm bypassed the shared identity rule, so between an outside relaunch and the
next read, a served read could address the dead process while claiming to observe the
live one. Reachability: an `open`-bound session whose app is quit and reopened by the
user leaves exactly this residue, and macOS `notRunningRefusal` never intercepts it.
The fix runs the shared rule on the arm before resolution, so identity is settled once
and both routes read the same live pid.

Pinned by a host-lane test that plants the residue (live app, bogus cached pid) and
asserts the served preparation refreshes without activating; canary: deleting the
refresh call fails the test with the bogus pid still in place. Also drops the probe
wrapper the arm no longer needs: a served read is one `state` query, and the
activating route's second read cannot reuse the first because the refresh between them
may have replaced the process the first answered for. The arm's and the predicate's
comments are cut to the invariant they protect, per AGENTS.md; the decision record
lives in #3338.

Partial #3254
…ents

AGENTS.md keeps decision and review narration out of implementation comments: the
screenshot raise site, `macAppCaptureNeedsRaise`, and the `.existingApp` policy case
each keep the one constraint they encode (region-grab pixels, the state rule, the
served-read scope) and point at #3338 for the measured record. No behavior change;
host-lane evidence for the pinned conditions is unchanged.

Partial #3254
@thymikee

thymikee commented Oct 9, 2026

Copy link
Copy Markdown
Member Author

All three items acted on; two commits at head e0bd78428, evidence below.

1. Stale pid on the served arm: real defect, confirmed from the caching path, fixed.
Reasoning, not assumption: resolveAppWithoutActivation returns mainOwned.app when the cache names the bundle, and the cache is only identity-checked by refreshCachedTargetIfProcessChanged — which the activating route runs before its targetNeedsActivation read (CommandDispatch ~583), while my arm skipped it entirely. So yes, the stale case is reachable on macOS: an open-bound session whose app is quit and reopened outside agent-device (user relaunch, crash-relaunch) leaves mainOwned.processIdentifier recording a dead pid while mainOwned.app/bundleId still match — the exact residue shouldRefreshCachedTarget exists to catch (pid-nil-guard aside, targetReset only covers the observed-reopen path). Between that relaunch and the next read, my arm would serve the dead handle as an ordinary read instead of the live process a bound read would have answered — worse than an activation, an unreported wrong-pid observation. The reviewer could not confirm the dead-handle read behavior; that uncertainty is why the fix runs the shared rule rather than special-casing: f6eafa3b1 calls refreshCachedTargetIfProcessChanged(bundleId:) on the arm before resolution, so both routes settle identity through one rule (file list: RunnerTests+CommandDispatch.swift, RunnerTests+CommandDispatchTests.swift). Pinned by testMacBackgroundReadRefreshesACachedHandleFromADeadPid, which plants the residue (live app, cached pid 999999) and asserts the served preparation refreshes to the live pid without activating; canary: deleting the refresh call fails it with ("Optional(999999)") is equal to ("Optional(999999)") — the bogus pid surviving into the served preparation.

2. Review history out of the comments: done in e0bd78428.
The screenshot site, macAppCaptureNeedsRaise, the .existingApp case, and the predicate doc are each cut to the one non-encodable constraint they protect (region-grab pixels; the state rule; served-read scope), pointing at #3338 for the measured record. Net −19 lines of prose. The testMacReadOfABackgroundAppIsServedInBackgroundWithoutActivating XCTAssertNil keeps the disclosure decision pinned where it's executable.

3. Second state query: confirmed, halved where it can be, kept where it can't.
The @MainActor wrapper did build a fresh probe on every macOS .existingApp command (including the ones falling through to activation), and prepareActivatedTarget then queried again via targetNeedsActivation. The wrapper is now gone — the arm inlines one query and a served read costs exactly one .state. The activating route's second query cannot be hoisted: refreshCachedTargetIfProcessChanged may replace the process between the two reads, so reusing the first answer would decide activation for a process that no longer exists — the same dead-pid class as item 1, moved from wrong-handle to wrong-decision.

Live-run runner.log (attached as requested). From state/sessions/mac3254/runner.log, run on the built runner from the tree that landed as 2df89243c (arm code identical at this head; arm + raise-gating unchanged by the two new commits). Session ran with Finder frontmost:

23:38:10.261  COMMAND_ACCEPTED command=snapshot  commandId=runner-c4c3426b…
23:38:10.644  COMMAND_COMPLETED command=snapshot commandId=runner-c4c3426b… ok=1
23:38:11.174  COMMAND_ACCEPTED command=snapshot  commandId=runner-465671bc…
23:38:11.420  COMMAND_COMPLETED command=snapshot commandId=runner-465671bc… ok=1
23:38:13.048  COMMAND_ACCEPTED command=screenshot commandId=runner-e7a4fcd6…
  t=2.83s Activate com.apple.systempreferences            ← window-level: raise, as designed
23:38:14.224  COMMAND_COMPLETED command=screenshot … ok=1
23:38:15.908  COMMAND_ACCEPTED command=screenshot commandId=runner-feebd871…   ← --fullscreen
23:38:16.170  COMMAND_COMPLETED command=screenshot … ok=1 ← no ACTIVATE line, 0.26 s, no settle
23:38:17.884  COMMAND_ACCEPTED command=tap commandId=runner-7d80dd7c…
23:38:17.886  AGENT_DEVICE_RUNNER_ACTIVATE bundle=com.apple.systempreferences state=3 reason=bundle_changed
23:38:17.954  AGENT_DEVICE_RUNNER_ACTIVATE_FACT bundle=com.apple.systempreferences reason=bundle_changed priorState=3 otherActiveApplicationPid=-
23:38:17.954  AGENT_DEVICE_RUNNER_FAST_APP_GUARD bundle=com.apple.systempreferences state=4

The two snapshot commands carry no AGENT_DEVICE_RUNNER_ACTIVATE* lines between ACCEPTED and COMPLETED; the first and only fact in the session is the tap's, with priorState=3. Frontmost stayed Finder across both reads and the fullscreen capture (osascript per step; full script + report at state/sessions/../report.txt in the run dir).

Unverified, staying honest: element.isHittable on an occluded background window — I have not measured it, and it predates this PR's snapshot path. The blast radius of a wrong hittable=true is bounded here because every action route still activates first (interaction foreground guard), so a ref click re-lands against the raised window; what stays unproven is whether the background tree's hittable/occlusion flags mislead inspection-only consumers. Flagging rather than claiming.

Validation at head e0bd78428: pnpm check:affected --run — all runnable checks passed. Host lane 295/295, 0 failures (selection tool: host lane reaches 295). Canary outputs quoted above. Nothing merged or approved.

@thymikee

thymikee commented Oct 9, 2026

Copy link
Copy Markdown
Member Author

This PR is ready. The code at e0bd784 is correct, and it fixes what the earlier review at 74f3899 left open. The served macOS arm now refreshes the cached target, and the screenshot and help text no longer overclaim.

All 19 checks pass at e0bd784, including the macOS runner lanes that exercise the changed served arm. There are no conflicts. Nothing else needs to happen before a human merges.

Not blocking, and you can take or leave it: in RunnerTests+CommandDispatchTests.swift (line 783) the planted cached handle is the live app and only the recorded pid is bogus, so the test proves the refresh runs on the served route but not that the served handle differs from a dead one. You could also assert prepared.app === mainOwned.app and that mainOwned.app is no longer the planted instance. Also, if the cache was bound with an unreadable pid, shouldRefreshCachedTarget returns false and the served arm skips the later targetNeedsActivation repair. I could not tell whether a bundle-id handle can address a dead process in that state.

I did not run the runner unit tests locally, and the canary failure on removing the refresh is as you reported it, though the code read supports it. The attached live runner.log is from before this change, so the refresh call on the served arm has no live run. The call is a non-activating cache rebind that the activating route already uses, and the macOS host-lane test exercises it through prepareActiveCommandContext, so I do not treat it as missing evidence.

On the other threads, the cubic-dev-ai threads are fixed at this commit and can be resolved. The APP_NOT_RUNNING claim was removed from the CLI help. The screenshot comment now limits the region-grab claim to window-level captures. The raise rows in the lifecycle tests match macAppCaptureNeedsRaise, and this change only touches that function's doc comment.

@thymikee
thymikee merged commit 903b0f4 into main Oct 9, 2026
19 checks passed
@thymikee
thymikee deleted the fix/macos-background-activation-3254 branch October 9, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant