Skip to content

Flaky e2e: ~5 chat.spec.ts tests are load-sensitive (shared-server / isolation), not a #61 regression #70

Description

@gbasin

Symptom

The chat e2e suite flakes rarely in CI. Concretely, the offline-send test turned master red once (96e4cc6) and passed on a clean re-run — i.e. a non-deterministic, load-sensitive flake, not a hard break. CI already cushions it (retries: 2 + trace: 'on-first-retry'), so it's low-frequency and non-blocking.

Investigation (evidence)

The load-sensitive tests

  • surface/e2e/tests/chat.spec.ts:235 — thread: reply in thread; root shows reply count
  • surface/e2e/tests/chat.spec.ts:299 — unread badge clears
  • surface/e2e/tests/chat.spec.ts:325 — cross-device read sync
  • surface/e2e/tests/chat.spec.ts:362 — search (⌘K) jump to message
  • surface/e2e/tests/chat.spec.ts:383 — offline send survives reload (the one that reddened master)

Root-cause hypothesis

The "fail together, pass alone" signature points at shared-server saturation and/or test isolation — 45 tests share one webServer (reuseExistingServer: false, started once) and one atrium_e2e DB reset per run. Under contention, heavy tests (offline + service worker + ws + reload + cross-context sync) exceed timeouts or see leaked state.

Recommended direction (not band-aids)

  • Root-cause from an actual CI failure trace (already captured on retry) rather than an overloaded local box.
  • Investigate per-test isolation (unique channels/users, not just unique text) + whether the shared server/DB leaks state across tests.
  • Consider sharding or serializing the heavy offline/disconnect/sync tests.
  • Avoid: more retries / blind timeout bumps (they'd mask the offline-durability behavior these tests legitimately guard).

Priority

Low / non-blocking — rare in CI, well-instrumented, no regression. Worth a deliberate stabilization pass if the flake rate becomes painful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions