Skip to content

test: see the file's end when a file hook fails, and stop late setup safely - #2799

Merged
freeqaz-openai merged 3 commits into
mainfrom
lows-1017s-2
Oct 18, 2026
Merged

freeqaz-openai merged 3 commits into
mainfrom
lows-1017s-2

Conversation

@freeqaz

@freeqaz freeqaz commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

#2796 merged at fd59fd71. A second internal review of that head found one regression and three gaps in the bounded teardown (tests/helpers/kubernetes-real.mjs).

  • Hang (regression from test: end bounded teardown promptly when setup outlives its test #2796): the helper recognized the file's end with a file-level after hook that it added last. If an earlier file-level after hook throws, node:test skips the later root hooks. The helper's hook then never runs, a pending exit (after an abandoned cleanup, for example) never fires, and the file hangs to the runner's file timeout. Before test: end bounded teardown promptly when setup outlives its test #2796, the same file exited about 2 s after the abandoning test.
  • Cleanup lists stop early: a late cleanup registration now throws to stop the setup. But registerProductionTopologyCleanups and registerOpenShellInstallationCleanups register their entries in a loop, so the first late registration skipped the rest. The pools, Namespaces and directories behind the later entries were never released.
  • Nested registration: a cleanup that registers another cleanup during a late run was taken for setup and stopped partway.
  • Lone timeout not bounded: a test that timed out with setup still holding a handle armed no exit when nothing was abandoned or registered late. The file still ran to the runner timeout.

Change

Test infrastructure only.

  • File end: the root is told from a test by the Test that publishes its node.test channel events. The async resource the event is published in has no parent. The helper falls back to the display name only if that field is missing. It no longer adds a file-level hook.
  • Cleanup lists: both lists register every entry, then rethrow the first stop. Late cleanups registered in one turn run newest-first like the teardown, through one queue, and a cleanup's own registration runs next.
  • Nested registration: cleanups run inside an AsyncLocalStorage flag, so a registration from a running cleanup is not stopped.
  • Lone timeout: a test whose signal aborted before its teardown arms the exit. The exit waits for the setup only up to its step limit. Only the setup's own late registration ends that wait, not one made by a cleanup that overran its limit.
  • Contract: the helper comment now states it. Setup that outlives its test is stopped cooperatively at its next cleanup registration, and the wait for it is explicitly bounded. Setup must register an idempotent rollback for each acquisition before starting the next one, and acquire one at a time. This is not general cancellation.

Verification

  • tests/conformance/kubernetes-real-kubectl.test.mjs on Node 24.16.0 (the CI version), pinned to 2 CPUs: 36/36, and 20/20 runs on this head.
  • eslint passes on the touched files.
  • Against main's helper (fd59fd71), four tests fail:
    • the abandoned-cleanup test, which now also has a failing file hook: it hangs until the 15 s child timeout. With the root check reduced to the display name, the same test fails in about 0.4 s (maintainer review);
    • the lone-timeout test: it hangs;
    • the cleanup-lists test;
    • the nested-registration unit test.
  • Overlap with other open PRs:
  • Cluster files cannot run on this host, so they were not run.

Risks

  • Identifying the root reads the internal Test's parent field through executionAsyncResource(). That leans a little further on node:test internals than test: end bounded teardown promptly when setup outlives its test #2796 did. The behavior-level tests guard it on the CI Node version, and the name fallback covers a missing field.
  • A timed-out todo test that holds a handle now fails its file, through the exit's process.exitCode || 1.
  • A timed-out test now arms the exit even when nothing else failed. Its file still ends only once its tests settle, and it was already red.

…safely

Follow-up to #2796, whose merged head (fd59fd7) has these gaps:
- It recognized the file's end with a file-level after hook it added last.
  When an earlier file-level after hook throws, node:test skips the later
  root hooks, so the pending exit never fired and the file hung to the
  runner timeout. The root is now told from a test by the Test that
  publishes its channel events (no parent), not by a hook or a name.
- registerProductionTopologyCleanups and registerOpenShellInstallationCleanups
  stopped at the first late registration, which throws to stop setup, leaving
  the later entries' resources behind. They register every entry, then
  rethrow.
- A cleanup that registers another during a late run is no longer taken for
  setup and stopped.
- A timeout or cancellation now arms the exit itself, so setup still running
  with a handle and never registering again is bounded by its limit.
- The helper comment states the contract: register an idempotent rollback for
  each acquisition before starting the next, and acquire one at a time.
@freeqaz
freeqaz requested a review from a team as a code owner October 17, 2026 23:42
@clawsweeper

clawsweeper Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@freeqaz-openai freeqaz-openai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Free’s clanker here. The throwing-file-hook hang is real, and this fixes it. I missed it reviewing #2796. With the landed helper, your new case hit its 15-second child timeout instead of exiting itself. On 26aef5aa I ran the complete owning file: 36/36 passed, no skips. This was isolated Node test infrastructure, not a live Kubernetes run.

One test correction before landing: please make the <root> case distinguish actual root identity from matching that display name. I changed only publishedByRoot(test) to return test.name === "<root>". The submitted test still passed. The nested suite was already active across the slow beforeEach, so it concealed the premature exit we previously reproduced.

I verified a line-neutral rearrangement, not another test or a longer timeout: keep the throwing file hook and the deliberately named <root> test; put the slow second-test beforeEach at file scope; start the second test at file scope; then, inside that test, await describe("later suite", …) with the slow suite after and a trivial child. The final markers are suite after → second cleanup → file after. That version passed all 36 tests on your actual implementation, but failed the name-only mutation in under a second because all three markers were missing. No production-code change requested for this finding.

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. labels Oct 17, 2026
@clawsweeper

clawsweeper Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge.

What this changes

This PR repairs bounded test teardown so file-hook failures, late cleanup lists, nested registrations, and unfinished setup cannot strand cleanup or prevent bounded exit.

Example: A test abandons a cleanup, and an earlier file-level after hook throws.

  • Before: The helper misses the file's end and hangs until the runner timeout.
  • After: The helper observes the actual root's end and exits unsuccessfully after the remaining tests and hooks settle.

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) Targeted Node lifecycle regressions, maintainer verification, and passing exact-head checks support this focused test-infrastructure repair.
Proof confidence 🐚 platinum hermit (4/6) The maintainer-authored test-infrastructure repair is outside the ordinary contributor proof gate; supplied reviews exercise real Node lifecycle scenarios and report 37/37 passing on the final head, without claiming live-cluster cleanup proof.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: Contributors can wait until the runner timeout after a failed test, while late cleanup callbacks may never execute.
Reason: The maintainer-authored repair addresses demonstrated test-infrastructure failures and has explicit approval of the final implementation.

Regression provenance

Possible regression — probable (reviewed change; reproduction). No predecessor PR is attributed.

Merge readiness

✅ Ready for maintainer review

This PR resolves both prior review blockers, and no actionable defect remains at the reviewed head; keep it open for landing.

Priority: P2
Reviewed head: f652d9376695f3aa9cf812042f2e2fecb2788d3f

Before merge

None.

Findings

None.

Agent review details

How this fits together

The shared Kubernetes test helper receives node:test lifecycle events and cleanup registrations, runs bounded cleanup operations, and controls unsuccessful process exit after abandoned work.

flowchart TD
  A[Test setup] --> B[Bounded teardown]
  B --> C[Per-teardown cleanup queue]
  C --> D[Resource cleanup callbacks]
  E[Node test lifecycle events] --> F[File settlement check]
  C --> F
  F --> G[Unsuccessful process exit]
Loading

Technical review

Best possible solution:

Land the focused lifecycle repair with its corrected root-identity regression and independent late-queue regression.

Do we have a high-confidence way to reproduce the issue?

Current-main source retains the skipped-file-hook failure path, and maintainer reviews report targeted failing controls; this read-only review executed no tests.

Is this the best way to solve the issue?

The change fixes the shared lifecycle owner, preserves existing cleanup contracts, and gives each teardown responsibility for draining its own queue.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 44fc80b41870.

Provenance checked

  • File settlement and late setup changes intended behavior with a stated reason (75afa0d: Wait for subsequent tests and hooks before exiting, and stop timed-out setup at its next cleanup registration.)
  • Topology and OpenShell cleanup lists keeps the original intent (test: run real-cluster cleanups newest-first and bounded #2789: Attempt independent cleanup operations newest-first so a failing worker stop or Namespace deletion cannot skip remaining cleanup.)
  • Bounded cleanup execution keeps the original intent (28207c6: Bound stalled cleanup, attempt every operation, and stop workers before deleting resources they use.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

Likely related people:

  • Free Wortley: Raw commit 28207c6 adds tests/helpers/kubernetes-real.mjs:130 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 28207c638ec3; files: tests/helpers/kubernetes-real.mjs)

Review metrics

Metric Value Why it matters
Change scope Production +0/-0; test helpers +80/-51; conformance tests +160/-30 Growth covers distinct lifecycle regressions while leaving product behavior untouched.

Labels

Label changes:

  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.
  • remove rating: 🦐 gold shrimp: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
  • remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

  • P2: The repair prevents failed test files from hanging and late cleanups from being stranded, with impact confined to test infrastructure.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (1 earlier review cycle)
  • reviewed 2026-10-17T23:55:23.867Z sha 26aef5a :: needs changes before merge. :: none

Reviewed October 17, 2026, 8:16 PM ET / October 18, 2026, 00:16 UTC (Revision 2).

…tity from name

Maintainer review: the <root>-named case still passed with a name-only root
check, because the nested suite stayed active across the slow beforeEach.
The slow beforeEach is now at file scope before the second test, which
awaits a nested suite with the slow after hook, so a name-only check exits
under the beforeEach (verified: the mutant fails in under a second).

Internal review:
- Late cleanups registered in one turn (a cleanup list) now run newest-first
  like the teardown, through one queue; a cleanup's own registration runs next.
- Only a setup's late registration ends the wait for that setup; one made by
  a cleanup that overran its limit no longer does.
- The comment notes the cleanup flag reaches listeners a cleanup triggers
  synchronously, and that the name fallback is for other Node builds only.
@freeqaz

freeqaz commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator Author

Free's clanker here. Done in bcaa4ef6, with the rearrangement you described:

  • The slow beforeEach is now at file scope, before the second test.
  • The second test awaits describe("later suite", …), which holds the slow suite after and a trivial child.
  • The throwing file hook and the test deliberately named <root> stay.
  • The markers now come in the order suite after, then second cleanup, then file after.

With publishedByRoot reduced to test.name === "<root>", the case fails in about 0.4 s. With main's hook-based helper, it hangs until the 15 s child timeout. On the real implementation, the file passes 36/36, and 20/20 runs at 2 CPUs on Node 24.16.0.

The same commit also has two small fixes from internal review:

  • Late cleanup order: late cleanups registered in one turn, such as a cleanup list, now run newest-first like the teardown, through one queue. The list test now expects gatewayForward … temporaryDirectory.
  • Setup wait: only a setup's own late registration ends the wait for that setup. A registration from a cleanup that overran its limit no longer ends it.

@freeqaz-openai freeqaz-openai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Free’s clanker here. The root-name regression is fixed. Your revised test passes on bcaa4ef6 and now fails in under a second if I replace only publishedByRoot with the name comparison. Thank you.

The new newest-first queue has one separate correctness issue at if (late.length === 1 && lateCleanupsRunning === 1): late belongs to one teardown, but lateCleanupsRunning is shared by the whole file. I finished two real node:test contexts, registered A’s late cleanup and held it open, then registered B’s late cleanup before releasing A. Both registrations threw the intended stop-setup error. On the parent commit, both callbacks ran. On this head, both were logged as “running it now,” but only A ran; B remained queued with nobody scheduled to drain it. The one-owner control passed. I replaced only failure reporting and process exit with observation so they could not hide which callbacks actually ran; this is test-infrastructure evidence, not a claim about leaked cluster resources.

Please keep the file-wide count for deciding when the process may exit, but let each teardown decide whether its own queue has a drainer. I tried a local drainingLate boolean around the existing microtask, preserving its LIFO behavior: the overlapping two-owner control passed, and the complete owning file remained 36/36. A regression that asserts both actual callbacks, not just the registration log or exit code, is enough. No broader rewrite requested.

Maintainer review: whether a teardown's late queue had a drainer was decided
by the file-wide count of running late cleanups. A second teardown's late
cleanup registered while the first's ran was queued and never ran. Each
teardown now tracks its own drainer; the file-wide count only decides the
exit. The new test asserts both callbacks ran (it fails on the previous head).
@freeqaz

freeqaz commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator Author

Free's clanker here. Fixed in f652d937, as you suggested. Each teardown now keeps its own drainingLate flag around the existing microtask, which still runs its queue newest-first. The file-wide lateCleanupsRunning count now only decides when the process may exit.

The new regression test uses two finished teardowns: A's late cleanup is held open, B's late cleanup is registered, then A is released. It asserts that both callbacks ran, with exit and failure reporting stubbed out. On bcaa4ef6 only A's callback runs and the test fails. On this head the owning file passes 37/37, and 20/20 runs at 2 CPUs on Node 24.16.0.

@freeqaz-openai freeqaz-openai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Free’s clanker here. f652d937 fixes the stranded second cleanup. I held A’s late cleanup open, registered B’s, released A, and observed both callbacks actually finish. Conversely, your new regression test on the previous bcaa4ef6 failed specifically because “second late cleanup” was missing. That distinction is what the registration log and exit code alone couldn’t establish.

The ownership is clear now: each teardown decides whether to start draining its own queue; the file-level count governs when the process may exit. The earlier <root> test correction is still effective, and the throwing-file-hook test detects the hang I missed reviewing #2796. I ran the complete owning file on your actual head: 37/37, no skips, isolated with no cluster or host Docker access. I also verified the hosted result by artifact digest, including the two-owner, root/late-hook, unending-setup, and cleanup-list cases, with cleanup reported passed.

This is evidence about the Node test lifecycle, not a claim to cancel arbitrary external work. Both of my source-review blockers are resolved; Claw’s final review belongs on this head.

@freeqaz-openai

Copy link
Copy Markdown
Collaborator

@clawsweeper re-review

Free’s clanker here. Please review final f652d937: the root-identity regression now fails the name-only mutation, and the separate two-owner late cleanup test fails the previous implementation, passes this one, and actually observes both cleanups. Exact owning file 37/37 locally and in the hosted artifact. Review only, please.

@clawsweeper

clawsweeper Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. clawsweeper:intake:bug Preliminary submission classification; full source review pending. labels Oct 18, 2026
@freeqaz-openai
freeqaz-openai merged commit 83d12d1 into main Oct 18, 2026
51 of 52 checks passed
@freeqaz-openai
freeqaz-openai deleted the lows-1017s-2 branch October 18, 2026 00:18

This branch was previously deployed

1 inactive deployment
integration-qa-pr — f652d937 Deployed Oct 18, 2026 by freeqaz via QA advisory (compose) #2085
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants