Symptom
The chat e2e suite flakes rarely in CI. Concretely, the offline-send test turned master red once (96e4cc6) and passed on a clean re-run — i.e. a non-deterministic, load-sensitive flake, not a hard break. CI already cushions it (retries: 2 + trace: 'on-first-retry'), so it's low-frequency and non-blocking.
Investigation (evidence)
The load-sensitive tests
surface/e2e/tests/chat.spec.ts:235 — thread: reply in thread; root shows reply count
surface/e2e/tests/chat.spec.ts:299 — unread badge clears
surface/e2e/tests/chat.spec.ts:325 — cross-device read sync
surface/e2e/tests/chat.spec.ts:362 — search (⌘K) jump to message
surface/e2e/tests/chat.spec.ts:383 — offline send survives reload (the one that reddened master)
Root-cause hypothesis
The "fail together, pass alone" signature points at shared-server saturation and/or test isolation — 45 tests share one webServer (reuseExistingServer: false, started once) and one atrium_e2e DB reset per run. Under contention, heavy tests (offline + service worker + ws + reload + cross-context sync) exceed timeouts or see leaked state.
Recommended direction (not band-aids)
- Root-cause from an actual CI failure trace (already captured on retry) rather than an overloaded local box.
- Investigate per-test isolation (unique channels/users, not just unique text) + whether the shared server/DB leaks state across tests.
- Consider sharding or serializing the heavy offline/disconnect/sync tests.
- Avoid: more retries / blind timeout bumps (they'd mask the offline-durability behavior these tests legitimately guard).
Priority
Low / non-blocking — rare in CI, well-instrumented, no regression. Worth a deliberate stabilization pass if the flake rate becomes painful.
Symptom
The chat e2e suite flakes rarely in CI. Concretely, the offline-send test turned
masterred once (96e4cc6) and passed on a clean re-run — i.e. a non-deterministic, load-sensitive flake, not a hard break. CI already cushions it (retries: 2+trace: 'on-first-retry'), so it's low-frequency and non-blocking.Investigation (evidence)
offline send survives reloadran 10/10 green locally (retries off).chat.spec.tstogether under CI env, 5 tests fail — they pass alone but destabilize when run together (shared webServer + a single DB reset per run, ~45 tests).cf7f79c(pre-feat(first-run): rebuild the first-time experience (8 directions) #61) vs96e4cc6(post-feat(first-run): rebuild the first-time experience (8 directions) #61) under identical conditions (CI=1, retries off,repeat-each 2, 1 worker). Identical 5-test failure set at both commits, withChat.tsxdiffering between them andchat.spec.tsunchanged. → the flakiness is pre-existing, not caused by feat(first-run): rebuild the first-time experience (8 directions) #61.The load-sensitive tests
surface/e2e/tests/chat.spec.ts:235— thread: reply in thread; root shows reply countsurface/e2e/tests/chat.spec.ts:299— unread badge clearssurface/e2e/tests/chat.spec.ts:325— cross-device read syncsurface/e2e/tests/chat.spec.ts:362— search (⌘K) jump to messagesurface/e2e/tests/chat.spec.ts:383— offline send survives reload (the one that reddened master)Root-cause hypothesis
The "fail together, pass alone" signature points at shared-server saturation and/or test isolation — 45 tests share one
webServer(reuseExistingServer: false, started once) and oneatrium_e2eDB reset per run. Under contention, heavy tests (offline + service worker + ws + reload + cross-context sync) exceed timeouts or see leaked state.Recommended direction (not band-aids)
Priority
Low / non-blocking — rare in CI, well-instrumented, no regression. Worth a deliberate stabilization pass if the flake rate becomes painful.