Skip to content

test: measure pointer gestures on the isolated visible display - #23678

Merged
nwparker merged 7 commits into
mainfrom
nwparker/ci-pointer-frame-budget
Sep 28, 2026
Merged

nwparker merged 7 commits into
mainfrom
nwparker/ci-pointer-frame-budget

Conversation

@nwparker

@nwparker nwparker commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 1 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​8 $\color{#cf222e}{\Huge{\mathbf{−}}}$​8 0
Prod 0 0 0 0

ELI5

The pointer test now measures gestures with a visible browser on CI's isolated display, where Chromium delivers input at the normal frame rate.

What Changed

Run the test in the existing electron-headful lane. Compare latency against the greater of the no-op RPC baseline and one 60 Hz frame, keeping the same 3/3/10 multipliers and all gesture checks.

Why

Hidden Linux windows delay button events by about one second and wheel events by about half a second. A no-op RPC also finishes faster than a frame, so it cannot alone represent the cost of rendered input. The isolated visible run measures real gestures without changing application behavior or weakening their timing requirements.

Linked Issue

CI failure: https://github.com/stablyai/orca/actions/runs/36476013862

Visual Proof

N/A — test configuration and timing baseline only.

Testing

  • Linux isolated-display E2E passes: move 16.2 ms, wheel 16.6 ms, click 32.6 ms; double-click press gap 33 ms.
  • Click, double-click, context menu, drag-selection, and targeted scrolling assertions retained.
  • PR CI validates the final head.

Review

No runtime, protocol, SSH, or product UI changes. Visible runs use CI's isolated display.

Agent skill upstream boundary

  • Not applicable.

Checklist

  • Small, focused, and self-reviewed.
  • Explained the failure and approach.
  • Platform and remote impact considered.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Frame-aware pointer budget — the spec now derives its reference from Math.max(latency.evalControl, 1000 / 60), so a sub-frame browser.eval control can no longer set an impossible budget, while the 3x/3x/10x multipliers and every gesture-fidelity assertion stay as they were.

The change is sound and the tradeoff is understood. On the fast macOS host where evalControl measured ~0.28–0.38 ms, the previous relative budget was below a single display frame, so the test rejected the correct in-process implementation. The floor keeps host-relative scaling on slow hosts and still catches the gross regressions the earlier Linux run showed (562 ms wheel, 2033 ms click). The deterministic no-subprocess guarantee lives in src/main/browser/agent-browser-bridge-pointer-input.test.ts:104, so the E2E spec remains a coarse latency budget rather than the correctness gate. No action needed.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@nwparker
nwparker changed the base branch from nwparker/fix-ci-e2e-09-28 to main September 28, 2026 19:58
@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 4002d069-7531-44e9-a0f9-80836bc741ac

📥 Commits

Reviewing files that changed from the base of the PR and between 9a2840b and 3602c92.

📒 Files selected for processing (1)
  • tests/e2e/browser-pointer-input-dispatch.spec.ts

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 9fac146e-f62b-41ba-9d2a-49caa3b0fcb2

📥 Commits

Reviewing files that changed from the base of the PR and between 55aea85 and 9a2840b.

📒 Files selected for processing (1)
  • tests/e2e/browser-pointer-input-dispatch.spec.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.


📝 Walkthrough

Walkthrough

The pointer latency assertions now use the greater of the measured evaluation-control latency and one 60 Hz frame as their baseline. The move and wheel limits remain three times the baseline. The click limit remains ten times the baseline.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 9a284

This test-only change accounts for frame pacing while preserving its measurements and latency limits. No actionable merge risk is evident; it is ready to merge subject to normal checks.

Architecture Summary

Architecture risk: 🔵 Low · up to 9a284

The change affects 1 system.

Changed systems: tests

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — tests (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in tests/e2e/browser-pointer-input-dispatch.spec.ts: Latency checks now use the greater of the eval control measurement and a 60 Hz frame duration as their baseline; move and wheel must remain below three times it, and click below ten times it. This replaces thresholds based solely on latency.evalControl.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Issue #23675 is closed and supplies historical context only. No active directly linked issue supplies coding requirements for this pull request.
Out of Scope Changes check ✅ Passed The reviewed change is limited to tests/e2e/browser-pointer-input-dispatch.spec.ts. It adjusts pointer latency test baselines for frame pacing and does not change production code. The change matches…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Title check ✅ Passed The title is concise, specific, and accurately describes the pointer gesture test change on the isolated visible display.
Description check ✅ Passed The description covers the change, reason, testing results, visual-proof rationale, and scope. It is mostly complete, although it does not provide a numbered issue reference and omits the template's N…
✨ Finishing Touches 💡 1
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nwparker nwparker changed the title test(e2e): account for frame pacing in pointer latency budgets test: measure pointer gestures on the isolated visible display Sep 28, 2026
@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[Low risk] Adjusts test thresholds for pointer gesture measurements.

The PR should not merge until the pointer-gesture test remains covered by recurring CI.

Findings

  1. P1 Gesture test leaves full CI ▶
  2. P2 Latency floor masks slower dispatch ▶

Summary

The PR moves the pointer-gesture test to the headful project and bases its latency limits on the greater of RPC control latency and one 60 Hz frame.

  • The headful run addresses hidden-window timing, but the test is absent from recurring full-suite CI.
  • The new latency floor reduces the test's ability to detect a slower dispatch path on fast hosts.

Reviews (1) · Last reviewed commit: "test: compare pointer timing on isolated..."

}

test('dispatches coordinate pointer input fast enough for real gestures', async ({ orcaPage }) => {
test('dispatches coordinate pointer input fast enough for real gestures @headful', async ({

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Gesture test leaves full CI

Adding @headful excludes this test from the headless project used by the full E2E CI shards and the default local E2E command. CI runs headful tests only when their spec is in the changed-spec list, so a full-suite run no longer checks pointer gestures or dispatch latency. Later regressions can go unnoticed unless someone explicitly runs the headful test.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +250 to +253
const dispatchControlMs = Math.max(latency.evalControl, 1000 / 60)
expect(latency.mouseMove, latencyReport).toBeLessThan(dispatchControlMs * 3)
expect(latency.mouseWheel, latencyReport).toBeLessThan(dispatchControlMs * 3)
expect(latency.click, latencyReport).toBeLessThan(dispatchControlMs * 10)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Latency floor masks slower dispatch

When the no-op control takes less than one frame, this floor raises the allowed pointer latency substantially. With a 5 ms control, a 30 ms move and a 103 ms click pass the new 50 ms and 167 ms limits, even though those times are roughly the slower helper-path multiples documented by the previous test. This makes the check less able to catch a return to that slower path on fast hosts.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Retagged the pointer-latency spec @headful — the test title now ends in @headful, moving it out of the electron-headless project (grepInvert: /@headful/) and into electron-headful (grep: /@headful/), where the fixture launches with ORCA_E2E_HEADFUL=1 and presents the window without focus. The frame-aware budget and all gesture-fidelity assertions are unchanged.

ℹ️ The latency budget now runs only in the changed-spec lane

Tagging @headful removes the spec from the scheduled --project=electron-headless shards (the only full-suite run), and the only scheduled headful invocation is the explicit worktree-switch-first-paint.spec.ts step at .github/workflows/e2e.yml:226. After merge this spec runs only when changed-e2e selects it, so the pointer-dispatch latency guard will no longer run on the daily reference suite. That may be the intended trade — the deterministic no-subprocess guarantee already lives in src/main/browser/agent-browser-bridge-pointer-input.test.ts:114 — but if the coarse gesture-latency budget should keep running routinely, it needs its own headful step alongside the first-paint benchmark.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@nwparker
nwparker merged commit b482b4d into main Sep 28, 2026
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant