Skip to content

Node 24 / V8 13 compatibility with CORO_PTHREAD coroutines - #5

Open
jbaczuk-qualia wants to merge 5 commits into
async-resourcefrom
node24-pthread
Open

jbaczuk-qualia wants to merge 5 commits into
async-resourcefrom
node24-pthread

Conversation

@jbaczuk-qualia

@jbaczuk-qualia jbaczuk-qualia commented Sep 21, 2026 •

Copy link
Copy Markdown

Summary

Makes fibers build and run on node 24 (V8 13.6) while keeping the CORO_PTHREAD coroutine backend that binding.gyp on async-resource already selects for every Linux build (and that arm64 production compiles today). Five commits on top of async-resource, nothing else:

Commit What Backend
33e592c compile against V8 13 and return the fiber's result from run() SetAccessor -> SetNativeDataProperty, Holder() -> This(), kFinalizer -> kParameter weak callbacks; uni::Return() takes PropertyCallbackInfo by const reference (V8 13 stores the argument slots inline, so the by-value copy silently dropped every getter result); fibers_async.js returns the wrapped function's value from runInAsyncScope any
5ecb0ab reset the handle when a suspended fiber is garbage-collected on V8 >= 10.4 kParameter is a phantom callback and V8 CHECKs that the handle was reset (Handle not reset in first callback); a yielded fiber whose JS object became unreachable aborted node 20+. Same change as #4, rebased any
d8ef8e2 forward Fiber.poolSize from the async wrapper to the native setter on V8 13, Fiber.poolSize = n on the wrapper creates an own property and the native pool size silently stays 120 (on node 18 the assignment reached the native setter). @qualia/patches relies on Fiber.poolSize = 1e9, so without this every finished coroutine beyond 120 concurrent ones was destroyed any (found on node 24)
d3260f6 destroy the coroutine context before freeing its stack ~Coroutine freed the stack and then called coro_destroy(). With pthread coroutines the thread's struct pthread lives at the top of that user-supplied stack, so pthread_cancel() dereferenced unmapped memory: SIGSEGV on every coroutine destroy. This is why test/pool.js and test/cleanup.js have segfaulted on every pthread build, node 18 included pthread (harmless elsewhere)
213c42d create bin/ recursively when installing from a source-only package build.js only created bin/<platform>/; published tarballs carry bin/ because of the prebuilts, npm pack of a source tree does not, so npm ci compiled fibers and then failed the final rename with ENOENT any

Not in this PR, on purpose (compare #2, the CORO_UCONTEXT port, now closed): no backend switch, no V8 ThreadId TLS key discovery, no v8_qualia_set_thread_stack_start hook. With pthread coroutines every fiber is a real OS thread, so V8's own per-thread state, its Locker/Unlocker thread archiving and cppgc's stack scan all work as designed and need no help from fibers or from V8 patches.

Node requirements

Runs on stock node 24 except for process.binding('async_wrap'), which stock node 24 refuses: setupAsyncHacks then disables itself and every yield corrupts the async hook stack (async hook stack has become corrupted, 7 of 19 suite failures on stock 24.21.0). qualialabs/node PR for custom-v24-pthread restores the allowlist entry and fixes node 24's thread_local TLS root cert store, which otherwise aborts on the first HTTPS request from a fiber thread. The Meteor server must run with --no-async-context-frame (Meteor's executionAsyncId() === 0 guard).

Verification

All on arm64 Linux (M-series container), node = qualialabs/node custom-v24-pthread (24.21.0 + 3 node-only patches, no V8 changes) unless stated.

Test Result
test/*.js (20 files incl. orphan-gc.js) 20/20. async-resource as shipped: 16/19 on node 18 pthread (pool.js, cleanup.js, future-exception.js)
pool.js / cleanup.js / orphan-gc.js before d3260f6 SIGSEGV in __pthread_cancel <- coro_destroy <- Coroutine::~Coroutine <- Coroutine::run (gdb, -g -O0 build)
d3260f6 applied to fibers 5.0.4 on node 18.20 (pthread) pool.js, cleanup.js and a 5-concurrent-coroutine destroy test pass; unpatched 5.0.4 segfaults on all three
GC stress: 8 fibers x 20 yields per round, heavy allocation, AsyncLocalStorage check, timer/immediate/nextTick resumes, --stress-incremental-marking --stress-compaction 100 rounds clean with AsyncContextFrame on, 100 with --no-async-context-frame, 300 with --stress-scavenge=50, 50 with poolSize = 4 (coroutines destroyed every round)
8 fibers each doing an HTTPS round trip that yields until the response, plus tls.createSecureContext({ca}) from 3 fiber threads pass (aborts in CleanupQueue::Add on stock node 24)
global-deployment-center in qli5 on a Graviton2 remote-dev host boots, HTTP 200, DDP login 403 + subscription ready, no failure signatures, 180 OS threads for the pooled fibers (see the comment below and qualialabs/qualia#56473)

Publishing

package.json still says 5.0.5, which is already published. Publish as 5.1.0 and pin it explicitly in node 24 services; ^5.0.4 consumers on node 18 keep 5.0.4 untouched. d8ef8e2 and d3260f6 also apply to node 18 and could go out as a 5.0.x later after a canary, separately from the node 24 work.

🤖 Generated with Claude Code

jbaczuk-qualia and others added 4 commits September 21, 2026 08:44
…om run()

- SetAccessor -> SetNativeDataProperty, Holder() -> This() (removed in V8 13)
- uni::Return takes PropertyCallbackInfo by const reference: since V8 13 the
  argument slots live inline in the struct, so a by-value copy lost every
  GetReturnValue().Set() and all accessors read as undefined
- fibers_async.js: return fn(...args) from runInAsyncScope so fiber.run()
  returns the function's result when the fiber finishes

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…V8 >= 10.4

f3cbba6 ("Make compatible with v20") registers the Fiber weak callback with
WeakCallbackType::kParameter because V8 removed kFinalizer. The two types have
different contracts: kFinalizer ran before the object was reclaimed and let the
callback resurrect it, which Fiber::WeakCallback relied on for suspended fibers
(ClearWeak(), unwind later in DestroyOrphans, MakeWeak() again). kParameter is a
phantom callback: the object is already gone and V8 CHECKs that the callback reset
its handle ("Handle not reset in first callback", global-handles.cc), so a yielded
fiber whose JS object becomes unreachable aborted the process on node 20+:

    # Fatal error in , line 0
    # Check failed: Handle not reset in first callback. See comments on |v8::WeakCallbackInfo|.

On V8 >= 10.4 the orphan branch now resets the handle in the callback and
DestroyOrphans deletes the fiber after unwinding it instead of re-weakening a handle
that no longer exists. MakeWeak(), ClearWeak() and the Fiber.current getter tolerate
the empty handle, which Fiber::Yield_ and JS code in the zombie's catch/finally
blocks hit while the stack unwinds (the first version without the guards segfaulted
in GlobalHandles::ClearWeakness). The node 18 (V8 10.2) code path is unchanged.

test/orphan-gc.js garbage-collects 200 yielded fibers, forces DestroyOrphans and
checks every fiber was unwound; it aborts on the unpatched build and passes here on
node 24.21.0 (patched custom-v24, ucontext) together with the other 19 tests, and
passes on node 18.16.1 with fibers 5.0.4.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
poolSize is a native data property on the native Fiber constructor. On V8 >= 13
(node 24) assigning `Fiber.poolSize = n` on the async wrapper no longer reaches that
setter: it creates an own data property on the wrapper and the native pool size
silently stays at its default of 120. On node 18 the same assignment did reach the
native setter. Qualia's @qualia/patches sets Fiber.poolSize = 1e9 so coroutines are
pooled forever; without this fix that became a no-op on node 24 and every finished
coroutine beyond 120 concurrent ones was destroyed (an OS thread per destroy with
CORO_PTHREAD, and the destroy path segfaulted, see the next commit).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…READ segfault)

Coroutine::~Coroutine freed the coroutine's stack and then called coro_destroy().
With CORO_PTHREAD the coroutine is an OS thread created with pthread_attr_setstack
on that stack, and glibc places the thread's `struct pthread` at the top of a
user-supplied stack. coro_destroy()'s pthread_cancel(ctx->id) therefore
dereferenced unmapped memory and the process died with SIGSEGV in
__pthread_cancel whenever a coroutine was destroyed, i.e. whenever more than
Fiber.poolSize coroutines finished. This is what has made test/pool.js and
test/cleanup.js segfault on every pthread build (node 18 included); it never
affected the ucontext/asm backends because nothing lives on their stacks after the
coroutine stops.

Destroy the context (cancel + join the thread) first, then free the stack. With
this, test/pool.js, test/cleanup.js and test/orphan-gc.js pass on CORO_PTHREAD.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…kage

afterBuild() renamed build/Release/fibers.node into bin/<platform>/ and only
created the last path component. Published tarballs always carried bin/ because
they ship prebuilt binaries in it, but `npm pack` drops empty directories, so a
source-only package (a vendored tarball of this branch, or a future publish
without prebuilts) compiled successfully and then failed the install with
ENOENT on the rename.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jbaczuk-qualia

Copy link
Copy Markdown
Author

GDC on the Graviton2 remote-dev host (2026-09-21, this branch + qualialabs/node#6 custom-v24-pthread binary, qli5, 4 cores): boots and serves (HTTP 200, Server startup complete, sync.data ran at boot), DDP login -> 403, meteor.loginServiceConfiguration subscription -> ready (2 runs), 26 yields observed in fiber_yield_switch_seconds, no Meteor code must always run within a Fiber, CleanupQueue::Add, async-hook-stack, Handle not reset or segfault lines in docker logs, 0 restarts after boot. Server child runs with --no-async-context-frame, 180 OS threads (one per pooled fiber), RSS 475 MB idle. Still to do: browser click-through, Lucky, --watch restarts, load. Details in gist da31bb93ceb6f57a66f1b47c1415dee7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant