Skip to content

Stabilize common package tests on CI - #7

Merged
miccy merged 2 commits into
sync/bun-migrationfrom
copilot/fix-ci-test-flakiness
Feb 3, 2026
Merged

Stabilize common package tests on CI#7
miccy merged 2 commits into
sync/bun-migrationfrom
copilot/fix-ci-test-flakiness

Conversation

Copilot AI commented Feb 3, 2026

Copy link
Copy Markdown

CI tests in packages/common/test were failing non-deterministically on Linux due to platform-specific behavior and async timing assumptions.

Changes

WebSocket.test.ts (line 382) - Platform-agnostic error assertions

  • Replaced conditional inline snapshots with invariant checks
  • Assert RetryError with WebSocketConnectionCloseError cause always present
  • Treat WebSocketConnectionError as optional (varies by platform/runtime)

Task.test.ts (line 729) - Eliminate abort/completion race

  • Added settled flag guarding both timeout and abort handlers
  • Ensures exactly one outcome recorded when disposal races with completion

Task.test.ts (10 locations) - Account for async task scheduling

  • Added await Promise.resolve() before assertions on events arrays
  • Allows microtask queue to flush before checking task start events
  • Applied at lines 3967, 5001, 5089, 5152, 5191, 5506, 5540, and others

Example fix:

// Before - flaky on CI
const fiber = run(parallel(all([task1, task2, task3])));
expect(events).toEqual(["start 1", "start 2", "start 3"]); // [] on Linux

// After - deterministic
const fiber = run(parallel(all([task1, task2, task3])));
await Promise.resolve(); // yield to microtasks
expect(events).toEqual(["start 1", "start 2", "start 3"]);

All changes scoped to test files only.

Original prompt

Fix CI test flakiness/failures in PR #6 by stabilizing non-production tests for Linux CI.

Context:

Observed failures (from logs):

  1. Inline snapshot mismatch in WebSocket test:
    • packages/common/test/WebSocket.test.ts around line 382: test "WebSocketConnectionError behavior on abrupt termination" expects platform-specific error list but CI receives different (e.g. only RetryError).
  2. Task abort/dispose behavior test mismatch:
    • packages/common/test/Task.test.ts:729 expected ["task1 aborted", "task2 aborted"] but received includes "task2 completed" (race between timeout completion and abort handler).
  3. Multiple assertions expecting immediate side-effects after starting parallel tasks:
    • Many expects like expect(events).toEqual(["start ..."]) receive [] on CI, indicating tasks start asynchronously (microtask/tick scheduling) in Linux/Bun/Vitest.
  4. Cascading timeouts in Task.test.ts (5000ms) likely caused by above scheduling/await issues.

Required changes (focus on making tests deterministic on CI; production code changes are optional but should be minimal):
A) Update WebSocket test to avoid snapshotting exact error arrays that differ by platform. Instead assert invariants:

  • closeCalled becomes true
  • mapped contains a RetryError with expected causeType (WebSocketConnectionCloseError)
  • for server or WebKit, mapped may also contain WebSocketConnectionError, but treat it as optional depending on runtime; do not rely on inline snapshot.
    B) Update Task dispose/abort test helper ("aborts all running fibers") to prevent race between timeout completion and abort:
  • introduce a settled flag guarding both the timeout callback and abort handler so only one outcome is recorded/resolved.
    C) Update tests that assert immediate events after starting parallel tasks to allow async start:
  • insert await Promise.resolve() (or await vi.waitFor(...)) before asserting events contains "start ...".
  • Apply to all failing sections mentioned in logs (around lines 3967, 4000, 4036, 4095, 5001, 5089, 5152, 5191, 5506, 5540) and any similar patterns.
    D) Keep changes scoped to test files under packages/common/test.

Acceptance criteria:

Files likely to edit:

  • packages/common/test/WebSocket.test.ts
  • packages/common/test/Task.test.ts

Use the PR title: "Stabilize common package tests on CI"

This pull request was created from Copilot chat.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: miccy <9729864+miccy@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix CI test flakiness in non-production tests for Linux CI Stabilize common package tests on CI Feb 3, 2026
Copilot AI requested a review from miccy February 3, 2026 07:38
@miccy
miccy marked this pull request as ready for review February 3, 2026 07:44
@coderabbitai

coderabbitai Bot commented Feb 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • 🔍 Trigger a full review

Comment @coderabbitai help to get the list of available commands and usage tips.

@miccy
miccy merged commit 76e2a8a into sync/bun-migration Feb 3, 2026
1 of 2 checks passed
@miccy
miccy deleted the copilot/fix-ci-test-flakiness branch February 10, 2026 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants