Skip to content

[Bug]: E2E suite is red on main — 22 of the last 22 scheduled runs failed #11149

Open

Description

@nwparker

Summary

The scheduled E2E suite has failed on every one of the last 22 scheduled runs on main. It is red right now, independent of any PR.

Surfaced while reviewing #11131 (adding a path-filtered E2E job to PR CI). That PR's own run failed 3 of 12 shards on specs unrelated to the change, which prompted checking the suite's baseline health.

Evidence

Recent scheduled runs on main, all failure:

2026-07-27T22:22:41Z  main  failure
2026-07-27T17:32:42Z  main  failure
2026-07-26T22:19:32Z  main  failure
2026-07-26T17:21:34Z  main  failure
2026-07-25T22:17:57Z  main  failure
2026-07-25T17:20:52Z  main  failure

Across the last 25 workflow runs, every schedule-triggered run (22 of them) failed. No scheduled green run in the queried window.

Failing specs observed on a recent run — spread across unrelated subsystems, so this looks like several independent failures rather than one root cause:

  • tests/e2e/agent-session-live-force-exit-resume.spec.ts — resumes a live agent record after force-exit restart
  • tests/e2e/agent-session-quit-resume.spec.ts — resumes an agent session after quit when its daemon PTY died
  • tests/e2e/artificial-opencode-terminal-load.spec.ts — keeps typing responsive while same-workspace panes redraw
  • tests/e2e/new-workspace-linked-item-project-switch.spec.ts — Jira issue stays linked / repo-scoped GitHub issue clears on project change
  • tests/e2e/plugin-panel-containment.spec.ts — contains hostile panel network, navigation, and bridge-flood probes
  • tests/e2e/terminal-opencode-altscreen-reveal-artifacts.spec.ts — reveal repaints a pane hidden inside an unclosed synchronized-output frame

Why it matters

E2E currently has no gate anywhere:

  • No PR trigger since #2098 (May 16) removed it from pr.yml.
  • Since #11031, the release path dispatches E2E after publication, where it cannot block.

So the only signal is the nightly schedule — and it has been red long enough that it no longer functions as a signal. A genuine regression landing today would be indistinguishable from the existing noise.

This is also why #11098 was able to merge with an e2e spec that failed deterministically on its own branch: nothing ran it.

Suggested next steps

  1. Triage the failures above — they appear to be several distinct issues, not one.
  2. Get one scheduled run green so the baseline means something again.
  3. Then flip ci(pr): run E2E when a PR touches tests/e2e paths (advisory) #11131's E2E job from advisory to blocking (a three-line change; the exact wiring is documented on verify's Require-successful-checks step in that PR).

Step 3 is deliberately gated on step 2: blocking on a red suite would prevent merging the very PRs that fix it.

Related: #10518, #11131, #11031, #11098

Made with Orca 🐋

Activity

  1. nwparker commented on Jul 28, 2026

    @nwparker
    ContributorAuthor

    #11131 is merged (1fa9ffb5ea), so E2E now runs and reports on PRs that touch tests/e2e/**, tests/playwright.config.ts, or .github/workflows/e2e.yml.

    It is deliberately advisory — e2e is not in verify.needs — precisely because of the failures tracked here. Gating on a suite that fails every scheduled run would block the PRs that fix it.

    Flipping to blocking once this is resolved is a three-line change, documented inline on verify's Require-successful-checks step in .github/workflows/pr.yml:

    1. add e2e to verify.needs
    2. add E2E: ${{ needs.e2e.result }} to that step's env
    3. after the existing loop, require "$E2E" to be success or skipped

    Step 3 has to sit outside the loop — skipped is the normal result for a path-filtered job, and folding that allowance into the loop would let a skip excuse the six always-required jobs. config/scripts/pr-e2e-gate-contract.test.mjs pins the current advisory state, so flipping it means updating that expectation too (intentionally — it makes the change explicit rather than silent).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions