Decide test quiescence from SwiftModel's own work registry, not from executor idleness

[mansbernhardt]

Replaces the way .modelTesting's wait verbs decide "is the model done?" — from observing the scheduler to asking SwiftModel's own registry of work. Design doc and its full reasoning are in #72; this is the implementation, built in four measured increments and landing behind a rule that cannot break anything that works today.

Why

expect / settle / waitUntil answered that question by counting outstanding executor jobs and checking queue idleness. That accounting is only as complete as our visibility into the Swift runtime, and the runtime exposes neither task state nor which executor a resumption travels through. So there is an open-ended set of ways work can be pending but invisible, and we have found and patched three separately: a task parked on a TestClock, a continuation hopping through the global executor after a yield, and a job sitting at background QoS. An open set cannot be closed by enumerating it, which is why this kept recurring.

SwiftModel already registers every node.task / forEach / onChange from creation to return. The drive just asked that registry the wrong question — "has it started" rather than "is it still running".

The rule

quiescent := oldSaysDone AND (newSaysDone OR runningWorkLooksUndeclared)

• oldSaysDone is today's predicate, unchanged and still required. That means the one residual case where the new answer could be a moment early — a third-party sequence delivering a value we cannot observe — can never cause a premature pass.
• newSaysDone — no registered work unit is running.
• runningWorkLooksUndeclared — every running unit has never parked and has shown no transition for the grace window. That is a compute loop or a sleep on a clock we cannot see into; the wait defers to the old answer, so behaviour is exactly what it is today, and a diagnostic names the task and its call site so the adoption is discoverable.
• A unit that has parked before, or transitioned recently, is mid-flight through a hop or a starved resumption, so the wait continues. That is the correctness fix, and the fallback is an all-quantifier precisely so it cannot swallow it.

Net effect measured across a suite run: the drive waits in 5 places where it used to conclude, the fallback fires twice, and everything passes.

What parks, and what adoption costs

node.forEach parks around its own next(), so any AsyncSequence — including swift-async-algorithms debounce / throttle — parks with no user action. SwiftModel's own streams (Observed, events, observeModifications) park at the source, so a hand-written for await behaves identically to forEach. For a bare clock.sleep, the clock wraps its own suspension in withModelParked { } — the adoption point is the clock implementation, not the call sites, which for a typical custom clock is a handful of edits. Docs/Testing.md documents it. Not adopting costs nothing: you get today's behaviour and a diagnostic.

Two production bugs found on the way

• A work unit's registration was released inside the innermost task-local scope, so it unregistered before catch handlers ran — and those write model state.
• Cancellations dropped entries before bodies unwound, so every teardown, task(id:) replacement and cancelPrevious swap looked quiescent while cancelled bodies were still running model-writing defers.

Corrections the design needed once it met real code

Recorded because each was found by probing rather than by argument: the running counter must start at registration, not body entry (which makes hasPendingStartTask fall out as a special case of the same rule); the park mark belongs to the input source, not to forEach's loop, since users write the loop by hand; the unpark must be done by the resumer, not the resumed task; the design's park-generation double-check was measured and is insufficient; and the eager unpark must sit at the raw source, because a filtered event resumes the consumer back into the same open park scope.

The last commit is the one to scrutinise, because it is not in the design: forEach runs the user's closure in the same unit as the loop that parked, so the rule as specified hung on node.forEach(stream) { await unadoptedClock.sleep(...) } — 22 ms to the trait cap. Inside a user closure the park history is now ignored and only the quiet window decides. Every framework stretch keeps the full guarantee. Nothing in this suite has that shape, which is why it needed probing rather than testing.

Gate

897 tests passing parallel and serial; 20 stress iterations clean; TSan per ci.yml clean with 896 tests and zero warnings; every suite log grepped for host crashes rather than trusting the summary line; release build warning-free; benchmarks flat (the ~3% hierarchy-mutation regression flagged mid-way was re-measured over 10 interleaved pairs and is noise). The quiescence suite itself went from 1–2 failures per 20 runs to 25 consecutive clean, after replacing three assertions of an unobservable transient state with a gated body that makes it durable.

Still open, deliberately

The foreign-delivery window (a third-party sequence resuming a consumer through internals we do not hold) is ~4 checks per run and needs the opt-in sequence wrapper from the design's hook 2. It cannot cause an early pass while oldSaysDone remains a required conjunct, which is why it does not block this. Roughly 4 checks per run are executor jobs owning no work unit, down from 31; the remainder looks like registered-but-not-yet-enqueued task creation.

🤖 Generated with Claude Code

[mansbernhardt]

Measured downstream and it is a large performance loss. Not merging this as it stands.

Paired, interleaved, both arms prebuilt at the same base and differing by exactly the quiescence change (A = 1.0.18 @ 27e6c6e, B = 448c637, both carrying the deadlock fix). Metric is summed per-test duration, not wall time, because wall time on that box swung 3m32s to 9m42s on identical code:

	1.0.18	#74
pair 1	13,429 s	53,883 s
pair 2	6,989 s	64,498 s

4× to 9× slower. The box was not quiet, so the magnitudes are soft, but both pairs agree on direction and the effect is an order of magnitude beyond the noise the paired design controls for. It is also exactly consistent with what this PR says it does: the new rule waits more, in five places per suite run in this repo's own suite and evidently far more in a large downstream one, and the cheaper quiescence check does not come close to paying for that.

The other motivation is also gone. The flakiness this was meant to address was the cross-tree dependency deadlock, fixed in 1.0.18 — so there is no flake rate left for a more truthful settle() to improve.

Staying open as a draft for the record, not as a queued change. What is worth keeping from it, independent of the rule:

• the demonstration that the old rule concludes while work is genuinely in flight, which is real and measured;
• two ordering bugs found on the way (a work unit unregistering before catch handlers run; Cancellations dropping entries before bodies unwind), both currently only observable through the new accounting;
• the design document in Sketch: semantic quiescence for .modelTesting (design proposal, no code) #72 and the corrections that came from building it.

If this is ever revived, the cost is the thing to attack first, and the suspect is the extra waiting rather than the accounting: each additional wait pays at least a grace window, and that is what a settle-heavy suite multiplies.

[mansbernhardt]

Closing: measured 4–9× slower downstream and the flake it targeted was the cross-tree deadlock fixed in 1.0.18. The two epilogue-ordering windows found on the way (dfbb85a) are only premature under this branch's registry-based rule — today's scheduler-observing rule counts the executor job as outstanding until the whole body, catch included, returns — so nothing carries over to main. Branch kept for the record.