Skip to content

Keep the zombie-cap test's outcomes off the scheduler - #213

Merged
myobie merged 1 commit into
mainfrom
agent/zombie-cap-test-ample-cap
Nov 4, 2026
Merged

myobie merged 1 commit into
mainfrom
agent/zombie-cap-test-ample-cap

Conversation

@myobie

@myobie myobie commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

This fixes a timing flake in operation::record::tests::an_observed_exit_is_gone_once_the_zombie_shows_within_the_cap. I found it while measuring the keyed flake for #212: it failed 1 of 60 whole pty-core lib runs at main, pinned to one core with two spinners.

Cause. The test gave liveness_after_exit_with a 5 ms wall-clock cap in every case. The loop reads the table, returns if it shows Gone or the cap has passed, and otherwise sleeps 200 µs. Two assertions therefore depended on how long those sleeps took:

  • The in-time case (record.rs:994) needs a third read before the cap. When a sleep ran past the cap, the loop correctly returned Unknown("exiting") after the second read, and the test expected Gone.
  • The still-exiting case (record.rs:986) asserted a second read. A single sleep after the first read can use up the whole cap.

Fix. Test only; liveness_after_exit_with is unchanged.

  • The in-time case is about stopping at the zombie, so it now gets an ample cap (10 s; it returns at the third read). It still asserts exactly three reads.
  • The still-exiting case keeps the 5 ms cap and its Unknown("exiting") and waited-out-the-cap assertions, and drops the read count. The in-time case already shows that the loop reads again.

Measured on Linux, at main 85b6ce5. Each run executes the test alone, 8 loops at once on one core with two spinners, 1200 runs per side. Main and this branch ran side by side:

failed at :994 (in time) at :986 (second read)
main, first pair 45/1200 44 1
main, second pair 49/1200 46 3
this branch 0/1200

The first pair's branch binary had only the ample cap, and the :986 assertion didn't fail in its 1200 runs. The second pair is the final change. Without contention, main failed 0/1000 and 0/1000 with the test alone, unloaded and on a loaded core. pty-core lib on this branch: 205 passed.

an_observed_exit_is_gone_once_the_zombie_shows_within_the_cap gave
liveness_after_exit_with a 5 ms wall-clock cap in every case. Two of
its assertions then depended on how long the 200 us sleeps between reads
took:
- The in-time case needs a third read before the cap. When a sleep ran
  past the cap, the loop correctly returned Unknown("exiting") after the
  second read, and the test expected Gone.
- The still-exiting case asserted a second read. One long sleep after the
  first read used the whole cap.

The in-time case is about stopping at the zombie, so it now has an ample
cap and still asserts exactly three reads. The still-exiting case keeps
the short cap and drops the read count; the in-time case already shows
that the loop reads again. liveness_after_exit_with is unchanged.
@myobie

myobie commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Independent read GO at exact head abe968e75e33d9ab76ee8f630d238068189ddc23. All 9 checks are SUCCESS on this head and the PR is CLEAN.

Read:

  • Test-only: one file (crates/pty-core/src/operation/record.rs, +7/-4); the code under test is unchanged. No wire-format change and no tier-1 file (events.jsonl, <id>.json, docs/disk-layout.md).
  • The flaky test is an_observed_exit_is_gone_once_the_zombie_shows_within_the_cap. It gave the post-exit re-read loop a 5 ms wall-clock cap, so on a loaded core a sleep between reads could outlast the whole cap.
  • The in-time case now uses a 10 s cap and still asserts exactly 3 reads, so it checks that the loop stops at the zombie. The still-exiting case keeps its check that it waited out the cap. It drops only its "read again in between" count, which the in-time case already covers.
  • Per the author, 1200 runs each on one contended core: the old test failed 45 and 49 times, the new one 0.

Risks (low): the still-exiting case no longer asserts how many times it re-read. A failure of the in-time case would now take up to its 10 s cap to report.

@myobie
myobie merged commit ddfcf09 into main Nov 4, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant