Clarify dual-lane macOS E2E policy#121
Conversation
📋 PR Overview
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe77812bf6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| ['visual report callouts', !hasScreenshots || /data-visual-annotation="report-callouts"/i.test(html)], | ||
| ['raw screenshot fallback', !hasScreenshots || /Open raw screenshot/i.test(html)], | ||
| ['peekaboo ax overlay labeled', !hasPeekabooAxOverlays || /Peekaboo AX overlay/i.test(html)], | ||
| ['no teammate-specific driver terminology', !TEAMMATE_SPECIFIC_DRIVER_PATTERN.test(html)], |
There was a problem hiding this comment.
When a PR title or head ref contains Scott (for example a branch like fkb/scott-peekaboo-local-e2e), the workflow forwards that metadata into the Peekaboo report and renderPeekabooPrFocus renders it in the HTML before verify-report runs. This new whole-document regex therefore fails the report for user-supplied PR metadata even though the generated driver terminology is clean, blocking an otherwise passing Peekaboo lane; the check should exclude dynamic PR metadata or scan only the report copy it owns.
Useful? React with 👍 / 👎.
1 participant
Summary
What changed operationally
peekaboo-e2e.ymlno longer downgrades an otherwise passing Peekaboo run toinconclusivesolely becausecu_covered != cu_required.run-local.mjsno longer downgrades Peekaboo single/suite reports toinconclusivesolely for missing Computer Use keys; self-tests now assert that missing breadth is reported as correspondence while failed Peekaboo-owned evidence still fails.Hygiene
.beadsruntime state is intentionally not included.Scott/scottteammate-specific terminology remains in tracked code/docs/report surfaces.native-driverwording was introduced; the lane is described as Peekaboo AX/screen-capture or shell/AX where mechanism matters.Verification
git diff --checknode --check tools/computer-use-e2e/run-local.mjsnode --check tools/computer-use-e2e/workflow-contract-self-test.mjsnode --check tools/computer-use-e2e/peekaboo-workflow-contract-self-test.mjsnode tools/computer-use-e2e/workflow-contract-self-test.mjsnode tools/computer-use-e2e/peekaboo-workflow-contract-self-test.mjsnode tools/computer-use-e2e/run-local.mjs self-testnode tools/computer-use-e2e/run-remote-cua.mjs self-testScott/scottreturned no matches.native-driverterminology returned no matches.Claude review was run before execution; the blocking feedback was to avoid framing this as docs-only because CI/report behavior changed. This PR uses
e2e:framing and calls the gate change out explicitly.