Background click via --app is acknowledged but has no effect on Chrome; profile-picker/menu items report zero-size bounds

[leeguooooo]

Version: peekaboo CLI (Homebrew, /opt/homebrew/bin/peekaboo), macOS 26 (Darwin 25.6), Chrome 152. Permissions: Screen Recording, Accessibility, Event Synthesizing all granted.

What happened:

1. peekaboo click --coords X,Y --app "Google Chrome" (the background/PID-targeted path) returned success (⏱️ Completed in 0.18s) but nothing happened in Chrome — tested on the "Who's using Chrome?" profile picker cards and on the toolbar extensions button. peekaboo click --coords X,Y --foreground at the same coordinates worked immediately. The docs say synthetic clicks "may be ignored by some apps until focus is allowed"; if the app ignores them, a success:false / outcome:unknown (or an automatic foreground fallback) would save a lot of blind retries.
2. peekaboo click --on <elem> --snapshot <id> on the Chrome toolbar "Extensions" button (elem_24, role button) also reported success:true twice while the popup never opened in the follow-up screenshot; perform-action --action AXPress on the same element returned success:false. A coordinate --foreground click opened it.
3. peekaboo see --app "Google Chrome" on the profile picker window returned the profile cards only as menuitem_* rows with bounds {x:0,y:0,width:0,height:0} (e.g. "立 (Leo)"), so there was no way to target a profile by element; I had to click by pixel coordinates from the screenshot.
4. peekaboo scroll has no --coords; it scrolls at the current mouse position, which is not documented on the command and surprised me when the pointer was elsewhere.

Expected: report failure when a PID-targeted synthetic click is dropped (or fall back to foreground), populate bounds for Chrome's profile-picker/menu items when available, and accept --coords on scroll.

filed by an AI agent with the user's consent

[clawsweeper]

Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed September 9, 2026, 2:51 PM ET / 18:51 UTC.

Summary
Keep open for the unresolved Chrome profile-picker observation. The collaborator’s native verification addresses the Extensions click and dispatch-reporting concerns, but explicitly does not establish that profile-card detection works; the repository’s conservative closure policy therefore does not permit closing this issue.

Reproducibility: no. high-confidence reproduction of the remaining picker defect is established. Native verification covered Extensions successfully, but neither that run nor this read-only source review exercised the reported picker state.

Ways to help us reproduce this

• Add a screenshot or short recording showing the behavior.
• Add expected vs actual behavior.

Next step
Obtain the exact current Peekaboo version and redacted picker screenshot, exact-window see JSON, and AX geometry/truncation evidence before assigning a repair; the active scope is profile-card observation.

Review details

Best possible solution:

Preserve explicit foreground consent and honest dispatch outcomes; correct profile-card observation only where native AX evidence establishes a Peekaboo defect, keeping coordinate scrolling a separate enhancement.

Do we have a high-confidence way to reproduce the issue?

No high-confidence reproduction of the remaining picker defect is established. Native verification covered Extensions successfully, but neither that run nor this read-only source review exercised the reported picker state.

Is this the best way to solve the issue?

Unclear until the picker’s native accessibility data is compared with Peekaboo output. The narrowed observation investigation is appropriate; automatic foreground fallback would conflict with the established delivery contract.

AGENTS.md: found and applied where relevant.

Remaining risk / open question:

• The remaining picker failure cannot yet be separated into missing Chrome accessibility data, observation truncation, or a Peekaboo traversal defect.

Codex review notes: model internal, reasoning medium; reviewed against 6cd38c0d319a.

Label changes

Label changes:

• remove clawsweeper:no-new-fix-pr: Current issue advisory state no longer selects this label.
• remove clawsweeper:needs-maintainer-review: Current issue advisory state no longer selects this label.
• remove clawsweeper:needs-product-decision: Current issue advisory state no longer selects this label.
• remove clawsweeper:needs-info: Current issue advisory state no longer selects this label.

Label justifications:

• P2: The remaining report concerns a bounded Chrome picker targeting problem with a working explicit foreground coordinate workaround.
• impact:ux-friction: Missing actionable profile-card observations force screenshot-based coordinate targeting and additional verification.

Evidence reviewed

What I checked:

• Native verification narrows the remaining report: The September 7 collaborator comment reports signed Peekaboo 4.3.1 and Chrome 152 verification: the Extensions popup opened after a background element click, the result exposed dispatched_unverified and retry-unsafe semantics, and app-only coordinates were refused. It expressly leaves profile-picker geometry unverified and treats coordinate scrolling as a separate enhancement. Background click via --app is acknowledged but has no effect on Chrome; profile-picker/menu items report zero-size bounds #687 (comment).
• Menu entries do not identify visible profile cards: Application-menu collection creates menuitem_ identifiers and substitutes zero bounds when frame information is unavailable. The detached observation path separately traverses the window and application menu bar, marking the latter applicationMenuBar; it does not establish that a similarly named menu entry represents a visible profile card. (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/MenuBarElementCollector.swift:94, 6cd38c0d319a)
• Current window geometry comes from native AX attributes: The detached worker reads AXPosition and AXSize, emits descriptors only when both dimensions exceed five points, and continues through children of omitted descriptors. Without the picker’s actual AX tree and observation diagnostics, this does not prove whether Chrome omits the cards or Peekaboo loses them. (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/DetachedAXObservationWorker.swift:643, 6cd38c0d319a)
• Click output preserves canonical outcomes: CLI regression coverage expects refused, partial, and indeterminate outcomes to return success:false, while dispatched_unverified retains success:true alongside its canonical outcome and target receipt. These tests were inspected, not executed. (Apps/CLI/Tests/CLIAutomationTests/ClickCommandActionResultTests.swift:17, 6cd38c0d319a)
• Latest release documents pointer scrolling: The v4.3.3 source documents physical-pointer foreground scrolling, background element targeting, and fresh-observation requirements. Current ScrollCommand metadata still has no coordinate option. This resolves the documentation concern without implementing the separate requested enhancement. (docs/commands/scroll.md:19, 46d7586c7f0e)
• Released outcome projection: The v4.3.3 projection exposes state, effect, retry_safe, and requires_fresh_observation. No release evidence inspected establishes a fix for the profile-picker state. (Core/PeekabooFoundation/Sources/PeekabooFoundation/DesktopActionOutcome+Projection.swift:68, 46d7586c7f0e)

Likely related people:

• steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[steipete]

Current native verification on macOS with the signed Peekaboo 4.3.1 release (f047bd97a) and Chrome 152.0.7977.76:

• A fresh AX-only observation of an exact Chrome window exposed the Extensions control as AXPopUpButton with a 34 × 34 frame. A background element click returned success: true together with state: dispatched_unverified, effect: unverifiable, retry_safe: false, and requires_fresh_observation: true. The next observation showed the Extensions heading and the button value changed to 1; the popup actually opened. Cleanup closed the popup and a further observation confirmed its heading was absent.
• App-only click --at 20,20 --app "Google Chrome" was refused before dispatch. Current background coordinate input requires the fresh exact-window screenshot receipt. No automatic foreground fallback occurred.
• Current scroll documentation explicitly states that foreground synthetic wheel events use the physical pointer, and describes the exact background element route. Coordinate scrolling remains a separate enhancement.

This narrows the unresolved portion to the profile-picker card geometry on the reported setup. The menuitem_ entries come from application-menu traversal and do not establish the geometry of visible profile cards. I have not reproduced that separate picker state and am not claiming it fixed. No input-policy change or automatic foreground fallback is proposed by this verification.

[steipete]

Verified the actual profile picker on macOS 26.6.2 (25G83), Google Chrome 152.0.7977.83, the signed Peekaboo 4.3.4 release (59a805b25), and a fresh Developer-ID-signed CLI + build-specific Bridge from clean main 1e23ab826593509bf0a9330386bd67f5c4b68607 (OpenClaw Foundation, FWJYW4S8P8). The test used a disposable Chrome user-data directory, opened the real Manage Chrome Profiles picker, and put Chrome behind another app; no CDP connection was opened.

The remaining behavior reproduces, but the native comparison establishes a Chrome AX limitation rather than a Peekaboo geometry conversion/hit-testing defect:

Native observation	Result
Cold picker, both AXUIElementCopyMultipleAttributeValues and individual AXUIElementCopyAttributeValue(AXChildren)	Both returned the same container hierarchy ending in an empty AXGroup; no profile-card node or geometry was available. Repeated Peekaboo observations at depth 60/max 5000 likewise contained only 12–13 window/container/titlebar elements.
After native hit-testing, Chrome exposed the embedded web subtree	Direct AX and fresh Peekaboo see both returned Open Your Chrome profile, AXButton, (1270,603,162,178); More actions for Your Chrome, (1404,607,24,24); Add, (1448,603,162,178); Guest mode, (968,956,148,36).
Hidden application-menu entry Profiles → Your Chrome	Direct AX itself returned position (0,1620) and size 0×0. This is an AXMenuItem, not the visible picker card; its name cannot supply the card's bounds.
Background press on the real card	Peekaboo returned state: dispatched_unverified, effect: unverifiable, retry_safe: false, requires_fresh_observation: true; the picker stayed open. A separate signed native probe calling AXUIElementPerformAction(card, AXPress) directly returned kAXErrorSuccess (0) and also left the picker unchanged.
Explicit foreground workaround	Focusing the exact picker window, then clicking the card's observed global point with --foreground --input-strategy synthOnly, closed picker window 119313; the selected profile's New Tab window 119294 became frontmost.

The original app-only coordinate command now returns success:false, state:refused, and mutation_dispatched:false before dispatch because a fresh exact-window screenshot receipt is required. The properly receipted cold-picker background coordinate attempt also returned success:false with the diagnostic “No pressable accessibility element was found”, conservatively marked indeterminate/retry-unsafe; it did not claim a completed click. A later accepted AXPress is deliberately reported as dispatched/unverified, because Chrome's success acknowledgement cannot establish that its handler ran.

The verified workaround is:

peekaboo window list --app "Google Chrome" --json
peekaboo see --window-id "$PICKER_WINDOW_ID" --no-elements --json
# Inspect that exact screenshot and map its window-local point to global logical points.
peekaboo window focus --window-id "$PICKER_WINDOW_ID" --pid "$CHROME_PID"
peekaboo click --at "$GLOBAL_X,$GLOBAL_Y" --global --foreground --input-strategy synthOnly --json
peekaboo window list --pid "$CHROME_PID" --json

If multiple Chrome instances share the application name, use --pid or --app "PID:<pid>"; the native test used that exact selector after also checking the literal --app "Google Chrome" command. Focus first matters: an initial combined exact-window coordinate click refused because Chrome did not report the picker as its AX focused window; the explicit window focus + global-click sequence above is the one actually verified. Do not retry an unverified action without observing again.

Closing with this evidence and workaround: Peekaboo cannot invent omitted native nodes, use a hidden menu item's zero-size frame as a card location, or turn Chrome's successful-but-ineffective AXPress acknowledgement into verified application behavior. Automatically activating Chrome would violate the explicit foreground contract. The existing click documentation covers sparse Chromium AX trees and foreground synthetic input; scroll documentation covers physical-pointer foreground scrolling and exact background element targeting, while coordinate scrolling is a separate feature rather than part of this bug. No product-code change or release is claimed, and the Chrome version that introduced this native behavior is unknown.

Validation: the clean-main CLI build passed, and the existing ClickCommandActionResultTests suite passed (including app, PID, snapshot, and coordinate receipt projections); the two opt-in ClickSnapshotWindowSelectionTests cases were skipped by their existing guard. Main also has green macOS CI and CodeQL at the exact source commit above.

Thanks @leeguooooo for the report—the distinction between menu entries and the actual picker cards was the missing native proof.

Inspected screenshots from the disposable profile

Before background input:

After a direct native AXPress returned success:

After explicit focus plus foreground input: picker closed and the selected profile's New Tab window became frontmost: