Exit vs futex block race reproducer update of sys exit#76
Draft
andreykarpenko-qc wants to merge 3 commits into
Draft
Exit vs futex block race reproducer update of sys exit#76andreykarpenko-qc wants to merge 3 commits into
andreykarpenko-qc wants to merge 3 commits into
Conversation
andreykarpenko-qc
force-pushed
the
exit_vs_futex_block_race_reproducer_update_of_sys_exit
branch
from
eaac1ba to
7567250
Compare
Add an h2 test that reproduces the exit() (whole-VM teardown) vs
futex-block race that the kernel_thread_stop_reap_blocked branch fixes,
made sweepable via a runtime spin parameter.
The race and why the fix is correct
-----------------------------------
Thread A calls exit(0) -> H2K_vm_stop -> vm_stop_locked: grabs the BKL,
sets vmblock->exiting = 1, reaps every non-RUNNING peer, then IPIs the
RUNNING peers (CLUSTER_RESCHED_INT) to self-reap. Thread B is a RUNNING
peer racing into H2K_futex_wait, where it grabs the BKL, removes itself
from the runlist and commits to H2K_STATUS_BLOCKED.
A's reap scan runs while B is still RUNNING, so reap_one_locked skips B
(the RUNNING case is a no-op). B only commits to BLOCKED after A drops
the BKL. Two mechanisms -- both new on this branch -- make that safe:
1. A's H2K_vmblock_finalize_if_done_locked early-returns while
num_cpus != 0. B's un-reaped slot keeps num_cpus == 1, so the
vmblock is NOT freed out from under B while B is still about to
dereference it. (Confirmed under lldb: at A's finalize the vmblock
shows num_cpus=1, status=0, exiting=1, and the freeing branch is
skipped.)
2. B is then reaped via the vmblock->exiting gate in resched():
reached either when B voluntarily blocks (H2K_dosched -> resched)
or via A's Pass-2 CLUSTER_RESCHED_INT. resched sees exiting,
self-reaps B and clears its context; finalize then completes
teardown once num_cpus reaches 0.
Without this branch's teardown machinery, exit() had no whole-VM reaper
at all (master has no H2K_vm_stop, no vmblock->exiting, no
reap_one_locked, and resched unconditionally requeues). A peer that
blocked on a futex during another thread's exit was stranded as a
permanently BLOCKED context on a VM that was supposed to be gone -- a
leaked / missed-wakeup thread -- and freeing the vmblock while B was
mid-commit would have been a use-after-free. The num_cpus guard in (1)
is exactly what closes that hazard.
Runtime spin parameter
-----------------------
A signed argv[1] selects which racing thread absorbs an artificial spin
delay, sliding one side's timing across the other's teardown so the
reproducer can land inside the race window:
argv[1] >= 0 : delay the BLOCKER by |argv[1]| spins (exiter tears down
immediately) so B commits to the futex wait late.
argv[1] < 0 : delay the EXITER by |argv[1]| spins so teardown lands
after B is already BLOCKED.
Default is 0 spins on both sides so the plain harness invocation still
runs; scripts/Makefile.inc.test now forwards ${GUEST_ARGS} to the guest
so a test's Makefile (or the command line) can pin the delay. On
v81/opt the tightest race -- A and B acquiring the BKL within one spin
iteration of each other -- sits at blocker_spins == 653 (bisected under
lldb: +652 the blocker wins, +653 the exiter wins); the large value is
because the exiter's puts("TEST PASSED") gives B a multi-thousand-cycle
head start.
scripts/testlist.v61: enable exit_vs_futex_block (v81 already had it).
Signed-off-by: Andrey Karpenko <andreyk@qti.qualcomm.com>
The default __h2_thread_stop_hook__ called pthread_exit, a top-of-stack API, from the lowest syscall layer -- a layering inversion that pulled the pthread runtime into every hosted link. Switch the default hook to the pure-h2 final exit primitive h2_thread_stop_trap(), matching what pthread_exit itself does on its last-thread path, and drop the pthread include. Also remove the (already-disabled) if (0 == status) guard so the hook fires on every exit. A nonzero exit status still reaches the harness via vmblock->status (set by both h2_vmkill and H2K_thread_stop_withlock, read back by the booter), so the ANGEL(SYS_EXIT) fall-through is not needed for failure reporting. Signed-off-by: Andrey Karpenko <andreyk@qti.qualcomm.com>
…elay The suite invoked the reproducer with GUEST_ARGS=0 (no artificial delay). Set it to 653, the v81/opt spin count that delays the blocker just enough for the exiter's teardown scan to catch it still RUNNING -- landing the run inside the exit-vs-futex race window rather than relying on incidental timing. Verified: blocker_spins=653, TEST PASSED, VM 2 status 0x0, exit 0. Signed-off-by: Andrey Karpenko <andreyk@qti.qualcomm.com>
andreykarpenko-qc
force-pushed
the
exit_vs_futex_block_race_reproducer_update_of_sys_exit
branch
from
7567250 to
a0eb3d1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.