Skip to content

fix: prevent scheduler deadlock when a ready task's delay expires mid-reschedule - #8

Merged
Shinrai merged 7 commits into
nextfrom
ci/isolate-bypass-delay-test
Aug 9, 2026
Merged

fix: prevent scheduler deadlock when a ready task's delay expires mid-reschedule#8
Shinrai merged 7 commits into
nextfrom
ci/isolate-bypass-delay-test

Conversation

@cldmv-bot

@cldmv-bot cldmv-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

🚀 What's Changed

💥 Breaking Changes

No breaking changes

✨ Features

No new features

🐛 Bug Fixes

📦 Dependencies

No dependency updates

🔧 Other Changes

👥 Contributors

…n file

The "bypassed task still applies its own completion delay" test hangs to the
30s vitest timeout on the lts/* CI matrix job every time it runs there, while
the explicit 20/22/24 jobs (same Node binary) pass. Doesn't reproduce locally,
including under CPU throttling. Moving it out of HoldMyTask.test.vitest.mjs
(72 tests, ~34 of which never call destroy() on their queue instance) into its
own file isolates it from any state/timer accumulation earlier tests in that
file might leave behind, to see if that's a factor.
@cldmv-bot cldmv-bot Bot added ! ci → next v4 flow: ci contributor PR targeting the next integration branch area: tests Touches test files, fixtures, or test infrastructure labels Aug 4, 2026
…-reschedule

_scheduleNextTick() only derived its next wake time from pendingHeap and
nextAvailableTime. If it ran at the exact moment nextAvailableTime had just
expired (a narrow race against schedulerTick's own timing) with pendingHeap
empty, nextTime fell through to Infinity, arming a ~24.8-day setTimeout and
stranding any task already sitting in readyHeap - a permanent hang, not a
slow test. This is what caused the bypassDelay + concurrency:1 + postDelay
tests to intermittently time out at exactly 30000ms on CI's lts/* job.
Reproduced locally via a stress loop (1 stall per ~6000 iterations); the
fix (accounting for a non-empty readyHeap when computing nextTime) ran
20000 iterations with zero stalls.
@cldmv-bot cldmv-bot Bot changed the title test(ci): isolate flaky bypassDelay completion-delay test into its own file fix: prevent scheduler deadlock when a ready task's delay expires mid-reschedule Aug 4, 2026
@cldmv-bot

cldmv-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Auto-normalized PR title: rewrote PR title to match the highest-priority commit type (fix).

  • Before: test(ci): isolate flaky bypassDelay completion-delay test into its own file
  • After: fix: prevent scheduler deadlock when a ready task's delay expires mid-reschedule

If this isn't what you want, edit the title — the normalizer won't re-fire as long as the title stays conventional.

@cldmv-bot cldmv-bot Bot added the area: core Touches core library / runtime source code label Aug 4, 2026
@Shinrai
Shinrai requested a lite review from Copilot August 4, 2026 14:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a scheduler edge case that could strand already-ready tasks by incorrectly scheduling the next tick far in the future, and adjusts the test suite to reduce CI flakiness by isolating a timing-sensitive test.

Changes:

  • Update traditional polling scheduler timing so ready-heap work triggers an imminent recheck instead of falling through to an effectively “never” timeout.
  • Move the bypassDelay completion-delay timing test into its own vitest file to isolate CI flakiness.
  • Remove the migrated timing test from the large combined HoldMyTask test file.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
src/hold-my-task.mjs Adjusts next-tick scheduling to avoid stranding tasks already in the ready heap.
tests/HoldMyTask.test.vitest.mjs Removes the bypassDelay completion-delay timing test from the main test suite file.
tests/BypassDelayCompletionTiming.test.vitest.mjs Adds an isolated timing test file to help diagnose/avoid CI-only hangs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/hold-my-task.mjs Outdated
Comment thread tests/BypassDelayCompletionTiming.test.vitest.mjs Outdated
Comment thread tests/BypassDelayCompletionTiming.test.vitest.mjs Outdated
Comment thread tests/BypassDelayCompletionTiming.test.vitest.mjs Outdated
…inity

Addresses PR #8 review feedback: the prior fix forced nextTime = now
whenever readyHeap was non-empty, which degrades an active post-completion
delay into a 25ms setInterval poll loop instead of a single setTimeout.
Narrow the guard to only fire when nextTime would otherwise fall through
to Infinity - the actual stranded-ready-task case. Verified: 20000-iteration
stress run stays at 0 stalls, and a normal delayed task now arms a single
timeoutId (not intervalId) again.

Also addresses three test-file review comments: a stale "priority 2" code
comment, wrap the isolated test's assertions in try/finally with `once`
instead of `on` so q.destroy() always runs even on assertion failure, and
a grammar fix in the file header comment.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/hold-my-task.mjs:1633

  • The new stranded-ready-task safeguard (forcing an imminent recheck when nextTime === Infinity && readyHeap is non-empty) fixes a subtle scheduler deadlock, but there’s no regression test covering the specific failure mode that motivated this change. Without a targeted test, it’s easy for future refactors of _scheduleNextTick() / schedulerTick() to reintroduce the 24.8‑day timeout behavior.

Consider adding a Vitest regression that reproduces the scenario (traditional polling mode) where a task is present in readyHeap while pendingHeap is empty and nextAvailableTime <= now, and assert that the scheduler continues processing (e.g., queue drains within a short timeout) rather than stalling.

		// already set nextTime above and should keep its single efficient timeout
		// rather than degrade into a tick-interval poll loop until it expires.
		if (nextTime === Infinity && this.readyHeap.size() > 0) {
			nextTime = now;
		}

Addresses PR #8 review feedback (suppressed comment on src/hold-my-task.mjs:1633):
no regression test covered the specific failure mode the fix addresses, so a
future refactor of _scheduleNextTick()/schedulerTick() could reintroduce the
24.8-day timeout fallback silently.

Uses the constructor's injectable `now` option to deterministically force the
exact race (a readyHeap task whose delay has just expired, with pendingHeap
empty) instead of relying on real wall-clock timing luck. Verified the test
fails against the pre-fix code (armed a ~2147483647ms setTimeout) and passes
with the fix (armed delay < 1000ms).
@Shinrai

Shinrai commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Addressing the suppressed comment from the 2026-08-07 review (src/hold-my-task.mjs:1633 — no regression test covered the stranded-ready-task failure mode):

Added a deterministic regression test in 964088d (tests/BypassDelayCompletionTiming.test.vitest.mjs). It uses the constructor's injectable now option to force the exact race (a readyHeap task whose delay has just expired, with pendingHeap empty) without relying on real wall-clock timing luck, then spies on setTimeout to assert _scheduleNextTick() arms an imminent recheck rather than the ~24.8-day fallback. Verified it fails against the pre-fix code and passes with the fix.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

tests/BypassDelayCompletionTiming.test.vitest.mjs:130

  • setTimeoutSpy.mockRestore() is called unconditionally right after _scheduleNextTick(). If _scheduleNextTick() ever throws (or future changes add a throw), the spy won't be restored and can leak into later tests. Also, restoring before you inspect mock.calls is brittle if Vitest ever changes the semantics of mockRestore().

Capture the calls before restoring, and restore the spy in a finally so cleanup is guaranteed.

		const setTimeoutSpy = vi.spyOn(global, "setTimeout");
		q._scheduleNextTick();
		setTimeoutSpy.mockRestore();

… restore

Addresses the suppressed Copilot comment on PR #8
(BypassDelayCompletionTiming.test.vitest.mjs:130): the regression test called
setTimeoutSpy.mockRestore() unconditionally right after _scheduleNextTick(),
so a throw there would leak the global spy into later tests, and it read
mock.calls after restoring. Now wrap the _scheduleNextTick() call in
try/finally, capture the recorded calls into a local before restoring, and
restore the spy in the finally so cleanup is guaranteed.
@Shinrai

Shinrai commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Addressing the suppressed comment from the 2026-08-08 review (tests/BypassDelayCompletionTiming.test.vitest.mjs:130setTimeoutSpy.mockRestore() called unconditionally, and mock.calls read after restore):

Fixed in 85afcf0. The _scheduleNextTick() call is now wrapped in try/finally; the recorded calls are captured into a local before restoring, and mockRestore() runs in the finally so the global spy can't leak into later tests even if _scheduleNextTick() throws.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

tests/BypassDelayCompletionTiming.test.vitest.mjs:68

  • q.destroy() is only guaranteed to run after the drain await resolves; if drain is never emitted (the exact failure mode described in this file’s header), the test will hang until Vitest’s global timeout and the queue won’t be cleaned up. Consider adding a per-test timeout around the drain wait and wrapping the wait + assertions in a single try/finally so cleanup always runs.
		await new Promise((resolve) => q.once("drain", resolve));

		try {
			expect(results).toEqual(["task1", "task2", "task3"]); // Normal priority order, task2 bypasses task1's delay

Addresses the suppressed Copilot comment on PR #8
(BypassDelayCompletionTiming.test.vitest.mjs:68): the `drain` await sat OUTSIDE
the try/finally, so if drain is never emitted - the exact deadlock this file
exists to catch - the test would hang to vitest's 30s global timeout and never
run q.destroy(), leaking timers. Moved the wait inside the try and raced it
against a 5s timeout that rejects with a clear message, so the failure surfaces
fast and cleanup always runs.
@Shinrai

Shinrai commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Addressing the suppressed comment from the latest review (tests/BypassDelayCompletionTiming.test.vitest.mjs:68 — the drain await sits outside the try/finally, so if drain never fires the test hangs to the 30s global timeout and never destroys):

Fixed in ff6c70f. Moved the drain wait inside the try and raced it against a 5s timeout that rejects with a clear message — so if drain never fires (the exact deadlock this file exists to catch), it fails fast well under vitest's global timeout, and q.destroy() in the finally always runs.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

tests/BypassDelayCompletionTiming.test.vitest.mjs:115

  • The regression test relies on a fixed 120ms sleep to assume task1 has completed and task2 has moved into readyHeap. On slower/contended CI, this can be nondeterministic and cause intermittent failures. Prefer waiting on an observable condition (e.g., results containing "task1") with a bounded timeout instead of a hard-coded delay.
	// Wait for task1 to actually run and complete (real time - the default poll
	// tick is 25ms, so this needs to clear a few real ticks; the queue's own delay
	// bookkeeping uses the injected now(), not real time).
	await new Promise((resolve) => setTimeout(resolve, 120));

…sleep

Addresses the suppressed Copilot comment on PR #8
(BypassDelayCompletionTiming.test.vitest.mjs:115): the regression test used a
hard-coded 120ms sleep to assume task1 had completed and task2 had reached
readyHeap, which is nondeterministic on slow/contended CI. Replaced it with a
bounded poll (2s deadline) on the observable condition - task1 in results and
task2 parked in readyHeap - and kept it inside the try so q.destroy() still runs
if it ever times out.
@Shinrai

Shinrai commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Addressing the suppressed comment from the latest review (tests/BypassDelayCompletionTiming.test.vitest.mjs:115 — fixed 120ms sleep is nondeterministic on slow CI):

Fixed in a456202. Replaced the hard-coded sleep with a bounded poll (2s deadline) on the observable condition — results contains task1 and task2 is parked in readyHeap (readyHeap.size === 1, pendingHeap.size === 0) — and kept it inside the try so q.destroy() still runs if it ever times out. Ran it repeatedly locally to confirm it's stable.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@Shinrai
Shinrai merged commit 8a22726 into next Aug 9, 2026
25 checks passed
@cldmv-bot
cldmv-bot Bot deleted the ci/isolate-bypass-delay-test branch August 9, 2026 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: core Touches core library / runtime source code area: tests Touches test files, fixtures, or test infrastructure ! ci → next v4 flow: ci contributor PR targeting the next integration branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants