feat(preemption): bound the preempt drain and diagnose a coroutine that never cooperates

AGENTS.md

@@ -133,6 +133,18 @@ The mechanism was selected by experiment, and the negative result is the load-be
   preemption did not ask to be interrupted and has no handler expecting it.
 - **The scheduler must hold a strong reference to every preempted fiber and drain them before
   shutdown.** Dropping one is fatal, and uninstalling the hook first does not help.
+- **The drain is bounded, and giving up on a coroutine is not letting go of it.** A coroutine with
+  no cooperative point at all is never drained, so `Scheduler::drainPreempted()` spends a budget
+  (64 resumes per coroutine, one second of wall clock per attempt, at least one resume each) and
+  then *reports* rather than spins. The straggler stays owned by the scheduler for the rest of the
+  process; the run raises `UndrainableCoroutineException` naming it and its spawn site, and the
+  preemptor ends the process with `posix_kill(self, SIGKILL)` from a shutdown function it registers
+  during shutdown, so every other shutdown function still runs first. **`exit()` is not an
+  alternative** — spike S7 measured it exiting 255 on the engine's fatal, because it still runs
+  request shutdown, which is where the fiber is destroyed.
+- **A drain may only resume while the slice timer is live.** A resume returns because the next tick
+  takes the CPU back, not because the coroutine hands it over. Draining with the clock disarmed is
+  the unbounded wait again, one step further along.
 - **Each forked worker re-arms its own timer** — `setitimer` intervals are cleared in the child.
 - **The clock is stopped for the idle poll, and the re-arm is a `finally`.** A free-running 10 ms
   timer wakes an idle preemptive process ~100 times a second to preempt nobody, so

README.md

@@ -516,6 +516,11 @@ z-engine requires it, and z-engine is a hard dependency of this package.
 - **Preemption is opt-in** (`new Runtime(preemptive: true)`) and, once armed, makes coroutine
   lifetimes the scheduler's business: a preempted coroutine is suspended inside an engine callback,
   so it is drained rather than discarded when a run ends.
+- **A coroutine with no cooperative point ends the process, with a diagnosis.** `while (true) { $x++; }`
+  never returns and never parks, so it can never be drained out of that callback and it can never be
+  released either. The drain gives it a budget, then `run()` throws `UndrainableCoroutineException`
+  naming the coroutine and the line that spawned it, and the runtime terminates the process itself
+  rather than leaving the fiber for the engine to destroy — which is an uncatchable fatal.
 - **An idle preemptive runtime is as quiet as a cooperative one.** The slice clock is stopped for
   exactly the time the process spends blocked in the poller — there is no coroutine to take the CPU
   away from — and started again on every way out of it, so a server waiting for work does not pay a

composer.json

@@ -25,7 +25,7 @@
     },
     "suggest": {
         "ext-pcntl": "Required for parallel workers (fork, signal handling) and for preemption",
-        "ext-posix": "Required for worker supervision"
+        "ext-posix": "Required for worker supervision, and for ending a run that a preempted coroutine refuses to leave"
     },
     "autoload": {
         "psr-4": {

spikes/README.md

@@ -17,6 +17,7 @@ run is in [`raw/`](raw/).
 | S4 | interrupt density in call-free loops | **GREEN**, with a hard caveat on single opcodes |
 | S5 | `Fiber::throw()` into a preempt-suspended fiber | **GREEN** — hazard established |
 | S6 | suspended-fiber GC | **GREEN** — with a shutdown obligation |
+| S7 | endings available with an undrainable fiber alive | **GREEN** (8.4) — only a signal avoids the S6 fatal |
 
 S1 being red is the load-bearing result: it rules out an FFI-free preemption path, so **preemption
 requires z-engine**, while Layer 1 remains FFI-free.
@@ -38,11 +39,12 @@ timeout 30 php8.4 -d ffi.enable=1 -d opcache.jit=off s4_interrupt_density.php
 
 Run each spike on **both** minors; a result that holds on one proves nothing about the other.
 
-### z-engine, for S2
+### z-engine, for S2 and S7
 
-S2 needs z-engine, and z-engine reads engine structures by byte offset, so the line must match the
-running minor. That means **two separate vendor trees** — one resolved by each PHP — which the
-scripts expect at `ze84/vendor` and `ze85/vendor`:
+S2 and S7 need z-engine, and z-engine reads engine structures by byte offset, so the line must match
+the running minor. That means **two separate vendor trees** — one resolved by each PHP — which the
+scripts expect at `ze84/vendor` and `ze85/vendor` (S7 falls back to the package's own `vendor/`,
+which is only correct for whichever minor that tree was resolved by):
 
 ```bash
 for v in 8.4:ze84 8.5:ze85; do
@@ -68,8 +70,10 @@ VERDICT S1: RED — Fiber::suspend() from handler raised FiberError: ...
 
 Verdicts are `GREEN`, `RED`, `HANG`, `CRASH`, `BLOCKED` or `INCONCLUSIVE`. Several scripts also take
 flags that deliberately trigger the failure they document (`--throw-probe`, `--unsafe-hook`,
-`--preempt-destroy`, `--preempt-shutdown`); those exit with a **PHP fatal error, by design** — see
-`VERDICTS.md` for the list and their survivable counterparts.
+`--preempt-destroy`, `--preempt-shutdown`, S7's `--leave-installed`, `--leave-uninstalled` and
+`--exit`); those exit with a **PHP fatal error, by design** — see `VERDICTS.md` for the list and
+their survivable counterparts. S7 runs all of its own modes as subprocesses, so running the script
+plainly is safe.
 
 A **segfault or bus error is never a flaky run.** Capture the command, the PHP version and a minimal
 reproducer, and report it.

spikes/VERDICTS.md

@@ -26,9 +26,16 @@ z-engine resolved per minor, exactly as the dependency policy requires:
 | **S4** interrupt density in call-free loops | no shape unbounded; worst non-allocating 92 µs | no shape unbounded; worst non-allocating 195 µs | **GREEN** (hard caveat: allocation + single opcodes) |
 | **S5** never `Fiber::throw()` into a preempt-suspended fiber | cancellation silently lost / fatal | cancellation silently lost / fatal | **GREEN** — hazard established |
 | **S6** suspended-fiber GC | 0 B/fiber leak; destroying a preempted fiber is **fatal** | 0 B/fiber leak; destroying a preempted fiber is **fatal** | **GREEN** — with a hard shutdown obligation |
+| **S7** endings available with an undrainable fiber alive | every shutdown path fatals; a self-directed signal does not | *not measured — see below* | **GREEN** on 8.4 |
 
 Nothing was BLOCKED: both z-engine lines installed successfully, so S2 was fully exercised.
 
+S1–S6 were run on both minors. **S7 was added later, from a session with a single vendor tree
+resolved by 8.4**, and its 8.5 column is therefore empty rather than assumed: it re-measures S6's
+fatal (identical on both minors there) and adds only which *endings* avoid it, which is a property of
+`fork`/`signal` semantics rather than of an engine offset. Re-run it on 8.5 with the `ze85` tree from
+[`README.md`](README.md) before treating the 8.5 column as known.
+
 ---
 
 ## S1 — `Fiber::suspend()` from a pcntl async signal handler
@@ -237,6 +244,43 @@ is inside the FFI callback, and the unwind is a non-`Throwable` engine sentinel
 the mandatory `catch (\Throwable)` cannot stop it. Uninstalling the hook does not help — the
 suspended fiber's *saved stack* still contains the ext-ffi trampoline frame.
 
+## S7 — endings available to a process holding an undrainable fiber
+
+> S6 says the drain is the only way out and issue #18 says the drain can never finish for
+> `while (true) { $x++; }`. Bounding it means deciding to stop while a fiber is still suspended in
+> the callback. What endings does the process have from there, and does any of them reach the end
+> without the engine destroying that fiber?
+
+**GREEN on 8.4 — exactly one family of endings avoids the fatal, and it is a signal.**
+
+Each row is a subprocess that preempt-suspends `while (true) { $x++; }` at a 2 ms slice, stops the
+timer, and then ends the way the row names (`raw/s7_php84.txt`):
+
+| ending | exit | S6 fatal? | output kept? |
+|--------|-----:|-----------|--------------|
+| let the script end with the fiber alive | 255 | **yes** | yes, then the fatal |
+| uninstall the interrupt hook first, then end | 255 | **yes** | yes, then the fatal |
+| `exit(70)` | **255**, not 70 | **yes** | yes, then the fatal |
+| `posix_kill(self, SIGTERM)` | 143 (signal 15) | no | yes, both streams |
+| `posix_kill(self, SIGKILL)` | 137 (signal 9) | no | yes, both streams |
+| kill from a shutdown function registered *during* shutdown | 137 (signal 9) | no | yes, and every earlier shutdown function ran first |
+| control: drain the fiber, then end normally | 0 | no | drained in **6 resumes** (2 M iterations at a 2 ms slice) |
+
+Three things this settles for the bounded drain:
+
+1. **`exit()` is not an escape.** It runs request shutdown, which is where the fiber is destroyed —
+   the process ends on the engine's fatal at 255 rather than on the code it was given.
+2. **A signal to self is.** The process ends where it stands, nothing is destructed, and everything
+   already written to stdout *and* stderr is kept — so the diagnosis survives the ending that
+   delivers it. `SIGKILL` over `SIGTERM` because a handleable signal can be handled by the
+   application, and this one may not be declined.
+3. **The kill can be deferred to the very last shutdown function.** Registering from inside a
+   shutdown function appends to the queue, so the runtime's ending does not swallow the
+   application's own shutdown work.
+
+The control row is also where the drain budget's size comes from: a coroutine that *does* finish
+needs a handful of resumes, not dozens.
+
 ---
 
 # Recommended preemption mechanism
@@ -305,6 +349,15 @@ is **not** covered here.
 - **Register a shutdown drain.** `register_shutdown_function()` runs early enough to drain
   preempted fibers safely (verified). Every preempted coroutine must be drained there before
   the engine destroys it.
+- **Bound the drain, and end the process yourself when it runs out.** A coroutine with no
+  cooperative point is never drained, so an unbounded drain is a hang. Stopping is safe only
+  because stopping is not releasing: the scheduler keeps holding the fiber, and the runtime ends
+  the process with `posix_kill(self, SIGKILL)` from a shutdown function registered during
+  shutdown. `exit()` is not an alternative — S7 measured it exiting **255 on the engine's fatal**,
+  not on the status it was given.
+- **A drain may only resume while the slice timer is live.** The resume returns because the next
+  tick takes the CPU back, not because the coroutine hands it over; draining with the timer
+  disarmed is the same unbounded wait in a different place.
 - **Cooperatively suspended fibers need no drain for memory.** 10 000 create/suspend/abandon
   cycles leak 0.00 B/fiber (`memory_get_usage(true)`), every destructor runs and every `finally`
   runs. The drain obligation is about the preempt path only.

spikes/raw/s7_php84.txt

@@ -0,0 +1,37 @@
+S7 — endings available to a process holding an undrainable fiber (PHP 8.4.19)
+
+--leave-installed          exit=255  signal=0  0.05s fatal=YES diagnosisKept=yes
+    | CHILD(--leave-installed): the fiber is preempt-suspended
+    | CHILD(--leave-installed): letting the script end with the fiber alive
+    | CHILD(--leave-installed): a shutdown function ran
+    | PHP Fatal error:  Throwing from FFI callbacks is not allowed in /tmp/claude-0/-home-user/a8c517fe-0774-4854-82ec-039954e12e12/scratchpad/wt-issue18/vendor/lisachenko/z-engine/src/System/Hook/InterruptHook.php on line 84
+--leave-uninstalled        exit=255  signal=0  0.05s fatal=YES diagnosisKept=yes
+    | CHILD(--leave-uninstalled): the fiber is preempt-suspended
+    | CHILD(--leave-uninstalled): hook uninstalled, letting the script end
+    | CHILD(--leave-uninstalled): a shutdown function ran
+    | PHP Fatal error:  Throwing from FFI callbacks is not allowed in /tmp/claude-0/-home-user/a8c517fe-0774-4854-82ec-039954e12e12/scratchpad/wt-issue18/vendor/lisachenko/z-engine/src/System/Hook/InterruptHook.php on line 84
+--exit                     exit=255  signal=0  0.05s fatal=YES diagnosisKept=yes
+    | CHILD(--exit): the fiber is preempt-suspended
+    | CHILD(--exit): calling exit(70) with the fiber alive
+    | CHILD(--exit): a shutdown function ran
+    | PHP Fatal error:  Throwing from FFI callbacks is not allowed in /tmp/claude-0/-home-user/a8c517fe-0774-4854-82ec-039954e12e12/scratchpad/wt-issue18/vendor/lisachenko/z-engine/src/System/Hook/InterruptHook.php on line 84
+--sigterm                  exit=143  signal=15 0.04s fatal=no  diagnosisKept=yes
+    | CHILD(--sigterm): the fiber is preempt-suspended
+    | CHILD(--sigterm): diagnosis on stdout before the signal
+    | CHILD(--sigterm): diagnosis on stderr before the signal
+--sigkill                  exit=137  signal=9  0.04s fatal=no  diagnosisKept=yes
+    | CHILD(--sigkill): the fiber is preempt-suspended
+    | CHILD(--sigkill): diagnosis on stdout before the signal
+    | CHILD(--sigkill): diagnosis on stderr before the signal
+--late-shutdown-function   exit=137  signal=9  0.04s fatal=no  diagnosisKept=yes
+    | CHILD(--late-shutdown-function): the fiber is preempt-suspended
+    | CHILD(--late-shutdown-function): registering from inside shutdown
+    | CHILD(--late-shutdown-function): a shutdown function ran
+    | CHILD(--late-shutdown-function): the late registration ran
+    | CHILD(--late-shutdown-function): killing from the late one
+--drain                    exit=0    signal=0  0.06s fatal=no  diagnosisKept=yes
+    | CHILD(--drain): the fiber is preempt-suspended
+    | CHILD(--drain): drained in 6 resume(s)
+    | CHILD(--drain): a shutdown function ran
+
+VERDICT S7: GREEN — leaving the fiber for request shutdown IS the S6 fatal; a self-directed SIGKILL after the diagnosis ends the process with the diagnosis intact and no fatal