fix(tts): stop x64 startup crash in ONNX NCHWc graph transformer - #39
Merged
Conversation
onnxruntime-node 1.23.2 segfaults inside NchwcTransformer while applying Level-3 (layout) graph optimisations to model_q8f16.onnx. The crash is a native abort inside InferenceSession.create, so the JS try/catch never runs and the process dies ~2-3s into startup with nothing logged and no crash report — the app just disappears. NCHWc is x86-only, which is why this reproduces on every x64 platform (Windows, Linux, Intel Mac) regardless of GPU vendor, and never on Apple Silicon. It also hits the CPU-only preload path, so --disable-gpu and acceleration settings make no difference. Cap graph optimisation at "basic" on x64 to skip the pass entirely. arm64 keeps ORT's default. Session init on arm64 is unchanged (~600ms either way). Fixes #36 Fixes #37 Fixes #38 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #36, fixes #37, fixes #38 — all three are the same bug.
Root cause
onnxruntime-node1.23.2 segfaults insideNchwcTransformerwhile applying Level-3 (layout) graph optimisations tomodel_q8f16.onnx. From the Intel Mac crash report in #38:It's a hard native abort (
EXC_BAD_ACCESS/0xC0000005) insideInferenceSession.create, so:try/catch→ CPU fallback increateSessionWithFallbacknever gets to run,%APPDATA%\Out Loudis never created,NCHWc is an x86-only transformer. That's why this reproduces on every x64 platform and never on Apple Silicon:
Loaded embedded model: …model_q8f16.onnx→Segmentation fault (core dumped)["cpu"]EPGPU vendor, DirectComposition, VC++ redistributable and Wayland/EGL are all red herrings — the crash also hits the CPU-only preload path, so
--disable-gpuand the acceleration setting make no difference.Fix
Cap
graphOptimizationLevelat"basic"on x64, skipping the Level-3 pass entirely. arm64 keeps ORT's default.Gating on
process.arch === "x64"rather thanprocess.platform === "win32": the crash is architecture-scoped, not OS-scoped, so a win32 gate would leave Intel Mac (#38) and Linux x64 (#37) broken.Graph optimisation only changes how the math is evaluated, not what it computes — worst case is last-decimal differences in output samples, nothing audible.
Verification
"basic"and"all": both succeed, init 591ms vs 613ms, so the arm64 path is unaffected and the option costs nothing measurable."disabled"→ OK,"basic"→ OK, mem-arena-off / single-thread → still crash. With"basic"the app reachesONNX session providers: cpu→Model preloaded successfullyand stays running.Credit for the root cause and the fix goes to @Blue4Life90, who diagnosed it down to the transformer and verified it on affected hardware.
Possible follow-up
"extended"(Level 2) should also avoid NCHWc — the layout transforms are Level 3, and this ORT version lists'layout'as a distinct level above'extended'— which would retain more optimisation than"basic". Nobody has verified that on x86 hardware yet, so this PR ships the tested value.Separately, this class of failure is invisible by design right now: three users lost days to a crash that writes no log. An early main-process log file (before
userDataexists) pluscrashReporteron Windows would make the next one self-reporting.🤖 Generated with Claude Code