Skip to content

fix(overlay): preload libstdc++ so webkit2gtk's JSC can't hijack std::call_once#214

Merged
srsholmes merged 3 commits into
mainfrom
fix/overlay-libstdcxx-preload-shim
Jul 15, 2026
Merged

fix(overlay): preload libstdc++ so webkit2gtk's JSC can't hijack std::call_once#214
srsholmes merged 3 commits into
mainfrom
fix/overlay-libstdcxx-preload-shim

Conversation

@srsholmes

Copy link
Copy Markdown
Owner

The radeonsi GL-init segfault, root-caused

This is the crash #211's zink fallback works around. Full chain, every link verified on the reproducing machine (CachyOS, Radeon 890M):

  1. Arch's webkit2gtk 2.52.5-1 (landed 2026-07-14, [extra]) regressed: libjavascriptcoregtk-4.1.so.0 now exports libstdc++'s private std::call_once machinery — __once_proxy (T), _ZSt11__once_call / _ZSt15__once_callable (B). In 2.52.3 they are U (imports, correct). Diffed directly between the two packages in the pacman cache.
  2. Electrobun's libNativeWrapper.so links webkit2gtk even in CEF mode (DT_NEEDED), so JSC enters the overlay process ahead of libstdc++ and wins interposition — LD_DEBUG=bindings shows even libstdc++'s own internal references binding to JSC's copies.
  3. Mesa's libLLVM (deepbind scope) binds the real libstdc++, so when radeonsi initializes its LLVM shader compiler (LLVMInitializeAMDGPUTargetstd::call_once), the writer stores the callback in libstdc++'s TLS slot and __once_proxy reads JSC's — NULLcall *0x0 → the exact Segmentation fault at address 0x0 from the reports.
  4. zink escaped only because ACO never calls LLVM. The kernel and mesa were bystanders: the crash timeline matched the 2026-07-14 system update because webkit2gtk 2.52.3→2.52.5 was in it.

GOT forensics for the record: at crash, libLLVM's __once_proxy GOT slot is correctly resolved to libstdc++ — the NULL is one level down, in the interposed TLS variable. gdb shows rip=0x0 reached from pthread_once internals.

Fix

A 9-line shim injected over the bun binary post-build (from inject-patched-wrapper.sh, which already runs from every build entry point): prepend libstdc++.so.6 to LD_PRELOAD and exec bun.real. The real libstdc++ then heads the global lookup scope and every object binds the same std::call_once state.

  • Why the shim wraps bun: the electrobun launcher replaces LD_PRELOAD for its child (verified: an inherited entry is dropped), so unit-level Environment= can never reach the process that matters. Upstream electrobun issue material.
  • Why not patchelf --remove-needed (tried): Zig emits eager GLOB_DAT relocations for address-taken webkit symbols — the load dies on undefined symbol: webkit_web_context_get_default. The clean fix (wrapper built without webkit in CEF mode) belongs upstream.
  • Bare soname → correct per-distro path (/usr/lib Arch, /usr/lib64 Fedora/Bazzite). Harmless where the bug is absent: libstdc++ is in the process either way. Idempotent injection; helpers inherit via env.

Verification (CachyOS / Radeon 890M / 100% repro machine)

Build radeonsi first start
without shim segfault ~200ms in, every time (57 panics this boot)
with shim NRestarts=0, ready flag written, #211's zink marker never arms, native radeonsi

Minimal-repro matrix that pinned the cause: plain eglinfo under the exact CEF preloads works; plain dlopen(libLLVM) + LLVMInitializeAMDGPUTarget() works; only the JSC-containing process crashes — and preloading libstdc++ cures it.

Relation to #211

Complementary, not competing. This removes the known trigger; #211's detect-and-retry remains the safety net for whatever regresses next (it caught this for a month). With both merged, affected machines stop crash-cycling entirely and stay on native radeonsi. Recommend #211's thread notes the root cause (comment incoming).

Follow-ups not in this PR: Arch bug for the webkit2gtk 2.52.5 symbol leak; electrobun issues for the LD_PRELOAD clobber and the unconditional webkit link.

🤖 Generated with Claude Code

…:call_once

Arch's webkit2gtk 2.52.5-1 regressed to exporting libstdc++'s private
std::call_once machinery (__once_proxy, __once_callable, __once_call)
from libjavascriptcoregtk; 2.52.3 imported them correctly. Electrobun's
libNativeWrapper.so links webkit2gtk even in CEF mode, so JSC lands in
the overlay process ahead of libstdc++ and wins interposition for
libstdc++'s own internal references.

That splits std::call_once's TLS state in two: mesa's libLLVM (deepbind
scope) writes the callback into libstdc++'s __once_callable, then
libstdc++'s __once_proxy reads JSC's copy — NULL — and the process
jumps to address 0x0. Trigger: radeonsi's LLVM shader-compiler init
during CEF GL bring-up, ~200ms after start, on AMD hardware. zink
escaped it only because ACO never touches LLVM.

Fix: wrap the bun binary in a shim that prepends libstdc++.so.6 to
LD_PRELOAD, restoring the real libstdc++ to the front of the lookup
scope so every object binds the same copy. Injected post-build from
inject-patched-wrapper.sh (runs from every build entry point).
Idempotent; bare soname so the loader picks the right per-distro path;
harmless on unaffected systems. The shim wraps bun itself because the
electrobun launcher REPLACES LD_PRELOAD for its child, so unit-level
env can never reach the process that matters.

patchelf --remove-needed on the wrapper does not work as an
alternative: Zig emits eager GLOB_DAT relocations for address-taken
webkit symbols, so the load fails on undefined symbols. The real fix —
rebuilding the wrapper without the webkit link in CEF mode — belongs
upstream in electrobun.

Verified on CachyOS / Radeon 890M (the machine that reproduces the
crash 100%): with the shim the overlay service comes up on native
radeonsi with NRestarts=0 and the #211 zink fallback never arms; the
identical build without the shim segfaults on first start.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@srsholmes

Copy link
Copy Markdown
Owner Author

Testing this fix on your machine — no merge, no release, no build needed

The fix is a small shell shim around the installed bun binary, so you can apply exactly what this PR ships to an existing install in under a minute, and roll it back with one command.

0. Confirm you're affected (10 seconds)

nm -D /usr/lib/libjavascriptcoregtk-4.1.so.0 | grep -E "once_proxy|once_call"
  • Lines showing T/B (e.g. T __once_proxy) → your webkit2gtk leaks the symbols; this fix applies to you.
  • Lines showing U or no output → you have a healthy webkit2gtk; your crash (if any) is something else.

Also check the crash signature matches: journalctl --user -u loadout-overlay -b | grep -m1 panic should show panic(main thread): Segmentation fault at address 0x0.

1. Apply the shim to your existing install

cd ~/.local/share/loadout-overlay/bin
mv bun bun.real
cat > bun <<'SHIM'
#!/bin/sh
export LD_PRELOAD="libstdc++.so.6${LD_PRELOAD:+:$LD_PRELOAD}"
exec "$(dirname "$0")/bun.real" "$@"
SHIM
chmod +x bun
# make sure the zink fallback marker isn't masking the test (harmless if absent)
rm -f "/run/user/$(id -u)/loadout-overlay-gl-fallback"
systemctl --user restart loadout-overlay

2. Verify

# 1 = healthy; the overlay should also actually open with your wake button
systemctl --user is-active loadout-overlay

# must be 0 new panics after the restart
journalctl --user -u loadout-overlay --since '-2min' | grep -c panic

# proves you're on NATIVE radeonsi, not the zink workaround:
# this file must NOT exist
ls "/run/user/$(id -u)/loadout-overlay-gl-fallback" 2>&1

Expected: service active, 0 panics, marker No such file or directory, overlay opens. Previously the first start always segfaulted ~200ms in.

3. Roll back (if anything is off)

cd ~/.local/share/loadout-overlay/bin
rm bun && mv bun.real bun
systemctl --user restart loadout-overlay

Notes

  • The shim is byte-identical to what this PR injects at build time — you are testing the real fix, not an approximation.
  • A Loadout reinstall/update will overwrite the manual shim (that's fine — once this PR is merged and released, the shipped build contains it).
  • Alternative for testing the full build pipeline instead: git checkout fix/overlay-libstdcxx-preload-shim && bun install && bun run build-and-install — heavier, same result.
  • Why no draft release: the Release workflow publishes to the fixed rolling tag that every curl | sh install resolves, so a branch build would overwrite the release for all users. The manual shim tests the same mechanism without that risk.

Please report back: your GPU, distro, the nm output from step 0, and whether step 2 came back clean.

🤖 Generated with Claude Code

…esting

One command takes a tester from an open PR to a running install:

    sh scripts/install-pr.sh 214

Fetches GitHub's refs/pull/N/head (works for fork PRs without knowing
the contributor's branch), shallow-clones into ~/.cache/loadout-pr-test/
so an existing checkout is never touched, then runs the standard
bun install --frozen-lockfile + build-and-install. Accepts a branch
name as well as a PR number; re-running re-fetches the PR's current
head. Reverting is the README install one-liner.

Deliberately not under ~/.cache/loadout: the backend runs as root and
owns that directory, so a user-run script cannot create anything there
(found the hard way — first test run failed on mkdir).

Verified end-to-end with PR #214 itself on CachyOS: fetched 36324fc,
built (inject step shimmed the bun binary as designed), installed;
overlay active on native radeonsi with zero panics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@srsholmes

Copy link
Copy Markdown
Owner Author

Easier way to test: install-pr.sh (now part of this PR)

This PR now ships scripts/install-pr.sh, so testing any PR pre-merge is one command. For this fix:

git clone git@github.com:srsholmes/loadout.git && cd loadout
git fetch origin pull/214/head && git checkout FETCH_HEAD -- scripts/install-pr.sh
sh scripts/install-pr.sh 214

(The git checkout FETCH_HEAD -- ... line is only needed until this PR merges — the script doesn't exist on main yet. Afterwards it's just sh scripts/install-pr.sh <N> from any checkout.)

What it does: fetches GitHub's refs/pull/214/head (fork PRs work too), shallow-clones into ~/.cache/loadout-pr-test/your existing checkout and install data are never touched — then runs the standard bun install --frozen-lockfile + bun run build-and-install. Sudo prompts once, at install time, same as the normal installer. Requires git + bun. Re-running the same command picks up new commits on the PR.

Verify after install:

systemctl --user is-active loadout-overlay                                   # active
journalctl --user -u loadout-overlay --since '-2min' | grep -c panic         # 0
ls "/run/user/$(id -u)/loadout-overlay-gl-fallback" 2>&1                     # No such file → native radeonsi

Revert any time with the README's install one-liner (reinstalls the official release over this).

Tested end-to-end with this very PR on the CachyOS repro machine: fetched 36324fc, the build's inject step installed the shim into the artifact as designed, and the resulting install runs on native radeonsi, zero panics — i.e. this comment's flow validates the fix through the full build pipeline, not just the manual shim from the earlier comment. The manual 30-second shim above remains the option for testers without bun.

🤖 Generated with Claude Code

…nstall-pr

- inject-patched-wrapper.sh: iterate find output line-by-line in both
  loops instead of word-splitting; the CEF tree already contains
  space-laden names and a space in any parent dir would have silently
  broken the loop. Verified against a 'dir with space' build tree,
  including idempotent rerun and empty-tree cases.
- install-pr.sh: default to the public https remote (no SSH keys
  needed by testers); keep origin in sync with LOADOUT_REPO when the
  cached workdir is reused.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@srsholmes srsholmes merged commit a393eca into main Jul 15, 2026
2 checks passed
@srsholmes srsholmes deleted the fix/overlay-libstdcxx-preload-shim branch July 15, 2026 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant