diff --git a/AGENTS.md b/AGENTS.md index 6f9213b..be6e237 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -133,6 +133,18 @@ The mechanism was selected by experiment, and the negative result is the load-be preemption did not ask to be interrupted and has no handler expecting it. - **The scheduler must hold a strong reference to every preempted fiber and drain them before shutdown.** Dropping one is fatal, and uninstalling the hook first does not help. +- **The drain is bounded, and giving up on a coroutine is not letting go of it.** A coroutine with + no cooperative point at all is never drained, so `Scheduler::drainPreempted()` spends a budget + (64 resumes per coroutine, one second of wall clock per attempt, at least one resume each) and + then *reports* rather than spins. The straggler stays owned by the scheduler for the rest of the + process; the run raises `UndrainableCoroutineException` naming it and its spawn site, and the + preemptor ends the process with `posix_kill(self, SIGKILL)` from a shutdown function it registers + during shutdown, so every other shutdown function still runs first. **`exit()` is not an + alternative** — spike S7 measured it exiting 255 on the engine's fatal, because it still runs + request shutdown, which is where the fiber is destroyed. +- **A drain may only resume while the slice timer is live.** A resume returns because the next tick + takes the CPU back, not because the coroutine hands it over. Draining with the clock disarmed is + the unbounded wait again, one step further along. - **Each forked worker re-arms its own timer** — `setitimer` intervals are cleared in the child. - **The clock is stopped for the idle poll, and the re-arm is a `finally`.** A free-running 10 ms timer wakes an idle preemptive process ~100 times a second to preempt nobody, so diff --git a/README.md b/README.md index e8eebd7..bae8091 100644 --- a/README.md +++ b/README.md @@ -516,6 +516,11 @@ z-engine requires it, and z-engine is a hard dependency of this package. - **Preemption is opt-in** (`new Runtime(preemptive: true)`) and, once armed, makes coroutine lifetimes the scheduler's business: a preempted coroutine is suspended inside an engine callback, so it is drained rather than discarded when a run ends. +- **A coroutine with no cooperative point ends the process, with a diagnosis.** `while (true) { $x++; }` + never returns and never parks, so it can never be drained out of that callback and it can never be + released either. The drain gives it a budget, then `run()` throws `UndrainableCoroutineException` + naming the coroutine and the line that spawned it, and the runtime terminates the process itself + rather than leaving the fiber for the engine to destroy — which is an uncatchable fatal. - **An idle preemptive runtime is as quiet as a cooperative one.** The slice clock is stopped for exactly the time the process spends blocked in the poller — there is no coroutine to take the CPU away from — and started again on every way out of it, so a server waiting for work does not pay a diff --git a/composer.json b/composer.json index 07aeba8..f6de7b4 100644 --- a/composer.json +++ b/composer.json @@ -25,7 +25,7 @@ }, "suggest": { "ext-pcntl": "Required for parallel workers (fork, signal handling) and for preemption", - "ext-posix": "Required for worker supervision" + "ext-posix": "Required for worker supervision, and for ending a run that a preempted coroutine refuses to leave" }, "autoload": { "psr-4": { diff --git a/spikes/README.md b/spikes/README.md index d707e39..09b0b6d 100644 --- a/spikes/README.md +++ b/spikes/README.md @@ -17,6 +17,7 @@ run is in [`raw/`](raw/). | S4 | interrupt density in call-free loops | **GREEN**, with a hard caveat on single opcodes | | S5 | `Fiber::throw()` into a preempt-suspended fiber | **GREEN** — hazard established | | S6 | suspended-fiber GC | **GREEN** — with a shutdown obligation | +| S7 | endings available with an undrainable fiber alive | **GREEN** (8.4) — only a signal avoids the S6 fatal | S1 being red is the load-bearing result: it rules out an FFI-free preemption path, so **preemption requires z-engine**, while Layer 1 remains FFI-free. @@ -38,11 +39,12 @@ timeout 30 php8.4 -d ffi.enable=1 -d opcache.jit=off s4_interrupt_density.php Run each spike on **both** minors; a result that holds on one proves nothing about the other. -### z-engine, for S2 +### z-engine, for S2 and S7 -S2 needs z-engine, and z-engine reads engine structures by byte offset, so the line must match the -running minor. That means **two separate vendor trees** — one resolved by each PHP — which the -scripts expect at `ze84/vendor` and `ze85/vendor`: +S2 and S7 need z-engine, and z-engine reads engine structures by byte offset, so the line must match +the running minor. That means **two separate vendor trees** — one resolved by each PHP — which the +scripts expect at `ze84/vendor` and `ze85/vendor` (S7 falls back to the package's own `vendor/`, +which is only correct for whichever minor that tree was resolved by): ```bash for v in 8.4:ze84 8.5:ze85; do @@ -68,8 +70,10 @@ VERDICT S1: RED — Fiber::suspend() from handler raised FiberError: ... Verdicts are `GREEN`, `RED`, `HANG`, `CRASH`, `BLOCKED` or `INCONCLUSIVE`. Several scripts also take flags that deliberately trigger the failure they document (`--throw-probe`, `--unsafe-hook`, -`--preempt-destroy`, `--preempt-shutdown`); those exit with a **PHP fatal error, by design** — see -`VERDICTS.md` for the list and their survivable counterparts. +`--preempt-destroy`, `--preempt-shutdown`, S7's `--leave-installed`, `--leave-uninstalled` and +`--exit`); those exit with a **PHP fatal error, by design** — see `VERDICTS.md` for the list and +their survivable counterparts. S7 runs all of its own modes as subprocesses, so running the script +plainly is safe. A **segfault or bus error is never a flaky run.** Capture the command, the PHP version and a minimal reproducer, and report it. diff --git a/spikes/VERDICTS.md b/spikes/VERDICTS.md index 619ab21..95ae18f 100644 --- a/spikes/VERDICTS.md +++ b/spikes/VERDICTS.md @@ -26,9 +26,16 @@ z-engine resolved per minor, exactly as the dependency policy requires: | **S4** interrupt density in call-free loops | no shape unbounded; worst non-allocating 92 µs | no shape unbounded; worst non-allocating 195 µs | **GREEN** (hard caveat: allocation + single opcodes) | | **S5** never `Fiber::throw()` into a preempt-suspended fiber | cancellation silently lost / fatal | cancellation silently lost / fatal | **GREEN** — hazard established | | **S6** suspended-fiber GC | 0 B/fiber leak; destroying a preempted fiber is **fatal** | 0 B/fiber leak; destroying a preempted fiber is **fatal** | **GREEN** — with a hard shutdown obligation | +| **S7** endings available with an undrainable fiber alive | every shutdown path fatals; a self-directed signal does not | *not measured — see below* | **GREEN** on 8.4 | Nothing was BLOCKED: both z-engine lines installed successfully, so S2 was fully exercised. +S1–S6 were run on both minors. **S7 was added later, from a session with a single vendor tree +resolved by 8.4**, and its 8.5 column is therefore empty rather than assumed: it re-measures S6's +fatal (identical on both minors there) and adds only which *endings* avoid it, which is a property of +`fork`/`signal` semantics rather than of an engine offset. Re-run it on 8.5 with the `ze85` tree from +[`README.md`](README.md) before treating the 8.5 column as known. + --- ## S1 — `Fiber::suspend()` from a pcntl async signal handler @@ -237,6 +244,43 @@ is inside the FFI callback, and the unwind is a non-`Throwable` engine sentinel the mandatory `catch (\Throwable)` cannot stop it. Uninstalling the hook does not help — the suspended fiber's *saved stack* still contains the ext-ffi trampoline frame. +## S7 — endings available to a process holding an undrainable fiber + +> S6 says the drain is the only way out and issue #18 says the drain can never finish for +> `while (true) { $x++; }`. Bounding it means deciding to stop while a fiber is still suspended in +> the callback. What endings does the process have from there, and does any of them reach the end +> without the engine destroying that fiber? + +**GREEN on 8.4 — exactly one family of endings avoids the fatal, and it is a signal.** + +Each row is a subprocess that preempt-suspends `while (true) { $x++; }` at a 2 ms slice, stops the +timer, and then ends the way the row names (`raw/s7_php84.txt`): + +| ending | exit | S6 fatal? | output kept? | +|--------|-----:|-----------|--------------| +| let the script end with the fiber alive | 255 | **yes** | yes, then the fatal | +| uninstall the interrupt hook first, then end | 255 | **yes** | yes, then the fatal | +| `exit(70)` | **255**, not 70 | **yes** | yes, then the fatal | +| `posix_kill(self, SIGTERM)` | 143 (signal 15) | no | yes, both streams | +| `posix_kill(self, SIGKILL)` | 137 (signal 9) | no | yes, both streams | +| kill from a shutdown function registered *during* shutdown | 137 (signal 9) | no | yes, and every earlier shutdown function ran first | +| control: drain the fiber, then end normally | 0 | no | drained in **6 resumes** (2 M iterations at a 2 ms slice) | + +Three things this settles for the bounded drain: + +1. **`exit()` is not an escape.** It runs request shutdown, which is where the fiber is destroyed — + the process ends on the engine's fatal at 255 rather than on the code it was given. +2. **A signal to self is.** The process ends where it stands, nothing is destructed, and everything + already written to stdout *and* stderr is kept — so the diagnosis survives the ending that + delivers it. `SIGKILL` over `SIGTERM` because a handleable signal can be handled by the + application, and this one may not be declined. +3. **The kill can be deferred to the very last shutdown function.** Registering from inside a + shutdown function appends to the queue, so the runtime's ending does not swallow the + application's own shutdown work. + +The control row is also where the drain budget's size comes from: a coroutine that *does* finish +needs a handful of resumes, not dozens. + --- # Recommended preemption mechanism @@ -305,6 +349,15 @@ is **not** covered here. - **Register a shutdown drain.** `register_shutdown_function()` runs early enough to drain preempted fibers safely (verified). Every preempted coroutine must be drained there before the engine destroys it. +- **Bound the drain, and end the process yourself when it runs out.** A coroutine with no + cooperative point is never drained, so an unbounded drain is a hang. Stopping is safe only + because stopping is not releasing: the scheduler keeps holding the fiber, and the runtime ends + the process with `posix_kill(self, SIGKILL)` from a shutdown function registered during + shutdown. `exit()` is not an alternative — S7 measured it exiting **255 on the engine's fatal**, + not on the status it was given. +- **A drain may only resume while the slice timer is live.** The resume returns because the next + tick takes the CPU back, not because the coroutine hands it over; draining with the timer + disarmed is the same unbounded wait in a different place. - **Cooperatively suspended fibers need no drain for memory.** 10 000 create/suspend/abandon cycles leak 0.00 B/fiber (`memory_get_usage(true)`), every destructor runs and every `finally` runs. The drain obligation is about the preempt path only. diff --git a/spikes/raw/s7_php84.txt b/spikes/raw/s7_php84.txt new file mode 100644 index 0000000..81ce761 --- /dev/null +++ b/spikes/raw/s7_php84.txt @@ -0,0 +1,37 @@ +S7 — endings available to a process holding an undrainable fiber (PHP 8.4.19) + +--leave-installed exit=255 signal=0 0.05s fatal=YES diagnosisKept=yes + | CHILD(--leave-installed): the fiber is preempt-suspended + | CHILD(--leave-installed): letting the script end with the fiber alive + | CHILD(--leave-installed): a shutdown function ran + | PHP Fatal error: Throwing from FFI callbacks is not allowed in /tmp/claude-0/-home-user/a8c517fe-0774-4854-82ec-039954e12e12/scratchpad/wt-issue18/vendor/lisachenko/z-engine/src/System/Hook/InterruptHook.php on line 84 +--leave-uninstalled exit=255 signal=0 0.05s fatal=YES diagnosisKept=yes + | CHILD(--leave-uninstalled): the fiber is preempt-suspended + | CHILD(--leave-uninstalled): hook uninstalled, letting the script end + | CHILD(--leave-uninstalled): a shutdown function ran + | PHP Fatal error: Throwing from FFI callbacks is not allowed in /tmp/claude-0/-home-user/a8c517fe-0774-4854-82ec-039954e12e12/scratchpad/wt-issue18/vendor/lisachenko/z-engine/src/System/Hook/InterruptHook.php on line 84 +--exit exit=255 signal=0 0.05s fatal=YES diagnosisKept=yes + | CHILD(--exit): the fiber is preempt-suspended + | CHILD(--exit): calling exit(70) with the fiber alive + | CHILD(--exit): a shutdown function ran + | PHP Fatal error: Throwing from FFI callbacks is not allowed in /tmp/claude-0/-home-user/a8c517fe-0774-4854-82ec-039954e12e12/scratchpad/wt-issue18/vendor/lisachenko/z-engine/src/System/Hook/InterruptHook.php on line 84 +--sigterm exit=143 signal=15 0.04s fatal=no diagnosisKept=yes + | CHILD(--sigterm): the fiber is preempt-suspended + | CHILD(--sigterm): diagnosis on stdout before the signal + | CHILD(--sigterm): diagnosis on stderr before the signal +--sigkill exit=137 signal=9 0.04s fatal=no diagnosisKept=yes + | CHILD(--sigkill): the fiber is preempt-suspended + | CHILD(--sigkill): diagnosis on stdout before the signal + | CHILD(--sigkill): diagnosis on stderr before the signal +--late-shutdown-function exit=137 signal=9 0.04s fatal=no diagnosisKept=yes + | CHILD(--late-shutdown-function): the fiber is preempt-suspended + | CHILD(--late-shutdown-function): registering from inside shutdown + | CHILD(--late-shutdown-function): a shutdown function ran + | CHILD(--late-shutdown-function): the late registration ran + | CHILD(--late-shutdown-function): killing from the late one +--drain exit=0 signal=0 0.06s fatal=no diagnosisKept=yes + | CHILD(--drain): the fiber is preempt-suspended + | CHILD(--drain): drained in 6 resume(s) + | CHILD(--drain): a shutdown function ran + +VERDICT S7: GREEN — leaving the fiber for request shutdown IS the S6 fatal; a self-directed SIGKILL after the diagnosis ends the process with the diagnosis intact and no fatal diff --git a/spikes/s7_undrainable_fiber_exit.php b/spikes/s7_undrainable_fiber_exit.php new file mode 100644 index 0000000..fd25e82 --- /dev/null +++ b/spikes/s7_undrainable_fiber_exit.php @@ -0,0 +1,340 @@ +new('struct itimerval'); + $value->it_interval->tv_sec = 0; + $value->it_interval->tv_usec = $usec; + $value->it_value->tv_sec = 0; + $value->it_value->tv_usec = $usec; + $libc->setitimer(0 /* ITIMER_REAL */, FFI::addr($value), null); + }; + + $want = false; + $executor = \ZEngine\Core::$executor; + $hook = \ZEngine\Core::setInterruptHandler(static function (object $hook) use (&$want): void { + try { + if ($want && \Fiber::getCurrent() !== null) { + $want = false; + \Fiber::suspend('PREEMPT'); + } + } catch (\Throwable) { + } + + try { + if ($hook->hasOriginalHandler()) { + $hook->proceed(); + } + } catch (\Throwable) { + } + }); + + pcntl_async_signals(true); + pcntl_signal(SIGALRM, static function () use (&$want, $executor): void { + $want = true; + $executor->requestInterrupt(); + }); + + register_shutdown_function(static function () use ($mode): void { + printf("CHILD(%s): a shutdown function ran\n", $mode); + }); + + $setInterval(PREEMPT_USEC); + + // The issue's coroutine, exactly: no park, no return, no cooperative point. + $runaway = new \Fiber(static function (): void { + $x = 0; + + while (true) { + $x++; + } + }); + + // For the control mode, a body that does finish once it is resumed enough. + if ($mode === '--drain') { + $runaway = new \Fiber(static function (): int { + $sum = 0; + + for ($index = 0; $index < 2_000_000; $index++) { + $sum += $index % 7; + } + + return $sum; + }); + } + + $suspendedWith = $runaway->start(); + $setInterval(0); + + if ($suspendedWith !== 'PREEMPT') { + printf("CHILD(%s): never preempt-suspended (got %s)\n", $mode, var_export($suspendedWith, true)); + exit(4); + } + + printf("CHILD(%s): the fiber is preempt-suspended\n", $mode); + + switch ($mode) { + case '--drain': + $resumes = 0; + + while (!$runaway->isTerminated()) { + $setInterval(PREEMPT_USEC); + $runaway->resume(null); + $setInterval(0); + $resumes++; + } + + printf("CHILD(--drain): drained in %d resume(s)\n", $resumes); + + break; + + case '--leave-uninstalled': + $hook->uninstall(); + printf("CHILD(--leave-uninstalled): hook uninstalled, letting the script end\n"); + + break; + + case '--leave-installed': + printf("CHILD(--leave-installed): letting the script end with the fiber alive\n"); + + break; + + case '--exit': + printf("CHILD(--exit): calling exit(70) with the fiber alive\n"); + exit(70); + + case '--sigterm': + case '--sigkill': + $signal = $mode === '--sigkill' ? SIGKILL : SIGTERM; + printf("CHILD(%s): diagnosis on stdout before the signal\n", $mode); + fwrite(STDERR, sprintf("CHILD(%s): diagnosis on stderr before the signal\n", $mode)); + flush(); + posix_kill(posix_getpid(), $signal); + + // Reached only if the signal did not end the process. + printf("CHILD(%s): SURVIVED the signal\n", $mode); + + break; + + case '--late-shutdown-function': + printf("CHILD(--late-shutdown-function): registering from inside shutdown\n"); + register_shutdown_function(static function (): void { + register_shutdown_function(static function (): void { + printf("CHILD(--late-shutdown-function): the late registration ran\n"); + fwrite(STDERR, "CHILD(--late-shutdown-function): killing from the late one\n"); + flush(); + posix_kill(posix_getpid(), SIGKILL); + }); + }); + + break; + } + + exit(0); +} + +// --------------------------------------------------------------------------- +// PARENT +// --------------------------------------------------------------------------- +/** @return array{stdout: string, stderr: string, exit: int, signal: int, seconds: float, timedOut: bool} */ +function s7_run(string $mode): array +{ + // The array form execs the binary directly: no shell in between to turn a signal death into an + // ordinary exit code and print "Killed" into the child's stderr. + $command = [PHP_BINARY, '-d', 'ffi.enable=1', '-d', 'opcache.jit=off', __FILE__, $mode]; + + $descriptors = [1 => ['pipe', 'w'], 2 => ['pipe', 'w']]; + $started = hrtime(true); + $process = proc_open($command, $descriptors, $pipes); + + if (!is_resource($process)) { + return ['stdout' => '', 'stderr' => 'proc_open failed', 'exit' => -1, 'signal' => 0, + 'seconds' => 0.0, 'timedOut' => false]; + } + + stream_set_blocking($pipes[1], false); + stream_set_blocking($pipes[2], false); + + $stdout = ''; + $stderr = ''; + $deadline = microtime(true) + CHILD_TIMEOUT; + $timedOut = false; + $status = proc_get_status($process); + + while ($status['running']) { + if (microtime(true) >= $deadline) { + $timedOut = true; + proc_terminate($process, SIGKILL); + + break; + } + + $read = [$pipes[1], $pipes[2]]; + $write = []; + $except = []; + + if (@stream_select($read, $write, $except, 0, 50_000) > 0) { + $stdout .= (string) stream_get_contents($pipes[1]); + $stderr .= (string) stream_get_contents($pipes[2]); + } + + $status = proc_get_status($process); + } + + $stdout .= (string) stream_get_contents($pipes[1]); + $stderr .= (string) stream_get_contents($pipes[2]); + + fclose($pipes[1]); + fclose($pipes[2]); + proc_close($process); + + return [ + 'stdout' => $stdout, + 'stderr' => $stderr, + 'exit' => $status['signaled'] ? 128 + $status['termsig'] : $status['exitcode'], + 'signal' => $status['signaled'] ? $status['termsig'] : 0, + 'seconds' => (hrtime(true) - $started) / 1e9, + 'timedOut' => $timedOut, + ]; +} + +printf("S7 — endings available to a process holding an undrainable fiber (PHP %s)\n\n", PHP_VERSION); + +if (s7_autoload() === null) { + echo "VERDICT S7: BLOCKED — no vendor/autoload.php carrying z-engine\n"; + exit(3); +} + +$results = []; + +foreach (CHILD_MODES as $childMode) { + $result = s7_run($childMode); + $results[$childMode] = $result; + $combined = $result['stdout'] . $result['stderr']; + + printf( + "%-26s exit=%-4s signal=%-2d %.2fs fatal=%s diagnosisKept=%s%s\n", + $childMode, + (string) $result['exit'], + $result['signal'], + $result['seconds'], + str_contains($combined, FFI_FATAL) ? 'YES' : 'no ', + str_contains($combined, 'preempt-suspended') ? 'yes' : 'NO ', + $result['timedOut'] ? ' TIMED-OUT' : '', + ); + + foreach (explode("\n", trim($combined)) as $line) { + if ($line !== '') { + printf(" | %s\n", $line); + } + } +} + +$kill = $results['--sigkill']; +$clean = !str_contains($kill['stdout'] . $kill['stderr'], FFI_FATAL) + && !$kill['timedOut'] + && $kill['signal'] === SIGKILL + && str_contains($kill['stdout'], 'diagnosis on stdout') + && str_contains($kill['stderr'], 'diagnosis on stderr'); + +$fatalOnLeave = str_contains($results['--leave-installed']['stdout'] . $results['--leave-installed']['stderr'], FFI_FATAL); + +printf( + "\nVERDICT S7: %s — leaving the fiber for request shutdown %s; a self-directed SIGKILL after the " + . "diagnosis %s\n", + $clean ? 'GREEN' : 'RED', + $fatalOnLeave ? 'IS the S6 fatal' : 'did NOT fatal (unexpected)', + $clean ? 'ends the process with the diagnosis intact and no fatal' : 'did not behave as required', +); + +exit($clean ? 0 : 1); diff --git a/src/Exception/UndrainableCoroutineException.php b/src/Exception/UndrainableCoroutineException.php new file mode 100644 index 0000000..a67c987 --- /dev/null +++ b/src/Exception/UndrainableCoroutineException.php @@ -0,0 +1,81 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + */ +declare(strict_types=1); + +namespace Lisachenko\NativePhpCoroutines\Exception; + +/** + * A preempted coroutine spent its whole drain budget without reaching a safe point. + * + * The scheduler resumes a preempt-suspended coroutine until it terminates or parks somewhere its + * own code chose, because a fiber suspended inside the interrupt callback cannot be released. A + * coroutine with no cooperative point at all — `while (true) { $x++; }` — never leaves the + * callback, and before the drain was bounded that was an unexplained hang at shutdown. + * + * This is that hang, converted into a sentence. It names each coroutine, how much of the budget it + * was given, and the line that spawned it, because the spawn site is the only thing that leads back + * to the loop that has to change. + * + * # Reading this exception means the process will not survive the run + * + * The straggler is still suspended in the callback and the scheduler still owns it: it is never + * dropped, because dropping it — or letting request shutdown destroy it — is + * `PHP Fatal error: Throwing from FFI callbacks is not allowed`, which no `catch` sees. Catching + * this exception buys the time to log it and nothing more; the runtime terminates the process from + * its shutdown handler once every other shutdown function has run. + */ +final class UndrainableCoroutineException extends \RuntimeException implements CoroutineException +{ + public const string HEADLINE = 'a preempted coroutine never reached a safe point - drain gave up!'; + + public const string REMEDY = 'give it a cooperative point - Coroutine::yield(), a channel, a ' + . 'sleep or a Context check - so the scheduler can resume it out of the preemption callback; ' + . 'a fiber left suspended in there can never be released, so the process is terminated ' + . 'rather than handed to the engine to destroy'; + + /** + * @param list $stragglers + */ + public function __construct(private readonly array $stragglers) + { + parent::__construct(self::HEADLINE . "\n" . self::renderDump($stragglers) . "\n" . self::REMEDY); + } + + /** + * The coroutines that would not cooperate, with the effort each one was given. + * + * @return list + */ + public function stragglers(): array + { + return $this->stragglers; + } + + /** + * @param list $stragglers + */ + private static function renderDump(array $stragglers): string + { + $lines = []; + foreach ($stragglers as $entry) { + $lines[] = sprintf( + 'coroutine #%d [resumed %d time(s) over %.3fs, still inside the preemption callback], ' + . 'spawned at %s', + $entry['id'], + $entry['resumes'], + $entry['seconds'], + $entry['origin'], + ); + } + + return implode("\n", $lines); + } +} diff --git a/src/Preemption/Preemptor.php b/src/Preemption/Preemptor.php index c090a38..d259106 100644 --- a/src/Preemption/Preemptor.php +++ b/src/Preemption/Preemptor.php @@ -13,6 +13,7 @@ namespace Lisachenko\NativePhpCoroutines\Preemption; use Lisachenko\NativePhpCoroutines\Coroutine; +use Lisachenko\NativePhpCoroutines\Exception\UndrainableCoroutineException; use Lisachenko\NativePhpCoroutines\Scheduler; /** @@ -69,12 +70,31 @@ * scheduler therefore keeps a strong reference to every preempted coroutine and drains it * ({@see Scheduler::drainPreempted()}), and this class registers a shutdown drain as the backstop * for a run that ends by panic or by `exit()`. + * + * # And the coroutine that will not be drained + * + * The drain resumes a coroutine until it returns or parks. One that does neither — + * `while (true) { $x++; }` — used to be resumed forever, which is a hang at shutdown with nothing + * to read. The drain is bounded instead, and this class owns what happens next: the straggler is + * still held, never released, and the process is ended deliberately with the diagnosis on STDERR + * ({@see self::endTheProcessWithADiagnosis()}). Being held and being drained are separate + * obligations; only the second one has a budget. */ final class Preemptor { /** The slice Layer 2 aims for, matching Go's own preemption interval. */ public const float DEFAULT_SLICE_SECONDS = 0.01; + /** + * The budget for a drain that is not allowed to give up, in seconds. + * + * A day, which no request shutdown outlives — "unbounded" written as a number, because the + * budget is compared against a monotonic clock and infinity does not survive the conversion. + * Only the ext-posix-less fallback uses it, where there is no way to end the process safely and + * spinning is the least-bad ending left. + */ + private const float UNBOUNDED_DRAIN_SECONDS = 86_400.0; + private readonly ItimerClock $clock; private readonly InterruptBridge $bridge; @@ -183,6 +203,10 @@ public function arm(): void * in a loop that never yields, and only a live timer guarantees that resuming it returns. That * is also why the idle pause is lifted before the drain rather than left to the ordinary path — * a teardown reached from inside a blocking poll would otherwise drain against a stopped clock. + * + * It is bounded, so this returns even for a coroutine that never cooperates. What it could not + * get out of the callback is still owned by the scheduler and reported by + * {@see Scheduler::undrainableCoroutines()}; disarming does not release anything. */ public function disarm(): void { @@ -399,6 +423,11 @@ private function onTick(int $signal, mixed $details = null): void * `Runtime::run()` drains on its own way out, so in an ordinary run this finds nothing to do. * It exists for the runs that never reach that code, where leaving one preempt-suspended fiber * for the engine to destroy is a fatal error with no catch clause anywhere. + * + * This is also the last place anything can be done about a coroutine that spent its budget + * without cooperating, which is why the straggler check lives here rather than only in + * `Runtime::run()`: the run may have ended by a panic, by `exit()`, or in a forked worker that + * never reaches that code at all, and the obligation is the same in every one of them. */ private function registerShutdownDrain(): void { @@ -415,6 +444,69 @@ private function registerShutdownDrain(): void $this->scheduler->drainPreempted(); $this->clock->disarm(); $this->armed = false; + + $stragglers = $this->scheduler->undrainableCoroutines(); + + if ($stragglers !== []) { + $this->endTheProcessWithADiagnosis($stragglers); + } + }); + } + + /** + * Say why, then take the process down before the engine can reach the fiber. + * + * Every ending available from here was measured (spike S7, `spikes/VERDICTS.md`), and the + * choice is forced: + * + * - letting request shutdown proceed with the fiber alive — `PHP Fatal error: Throwing from + * FFI callbacks is not allowed`, exit 255, uncatchable, on both minors; + * - uninstalling the interrupt hook first — the same fatal, because it is the fiber's *saved + * stack* that carries the FFI trampoline frame; + * - `exit()` — the same fatal, because it still runs request shutdown; + * - a signal to self — the process ends where it stands, no fiber is destroyed, and everything + * already written is kept. + * + * So the ending is a signal, and it is `SIGKILL` rather than a politer one because a signal + * that can be handled can be handled by the application, and this one may not be declined: the + * only alternative to it is the fatal above. It is sent from a shutdown function registered + * *from inside* this one, which PHP appends to the queue, so every other shutdown function the + * application registered still runs first (S7 `--late-shutdown-function`). + * + * The diagnosis goes to STDERR because at this point in shutdown output buffering may already + * be gone and the exception can no longer be thrown anywhere anybody could catch it. + * + * Without ext-posix there is nothing to send the signal with, and the drain goes back to being + * unbounded — a diagnosed wait instead of a silent one. That is the worse ending of the two on + * purpose: "never release a preempt-suspended fiber" is an invariant, and waiting is the only + * thing left that keeps it. + * + * @param list $stragglers + */ + private function endTheProcessWithADiagnosis(array $stragglers): void + { + $diagnosis = new UndrainableCoroutineException($stragglers); + + if (!function_exists('posix_kill')) { + fwrite(STDERR, $diagnosis->getMessage() . PHP_EOL + . 'ext-posix is not loaded, so this process cannot be ended safely; draining the ' + . 'coroutine instead, which will not return until it cooperates' . PHP_EOL); + + // Re-arming is not optional here: the timer is the only reason a resume ever comes + // back, and this drain deliberately has no budget to stop it. + $this->arm(); + $this->scheduler->drainPreempted(self::UNBOUNDED_DRAIN_SECONDS, PHP_INT_MAX); + $this->disarm(); + + return; + } + + register_shutdown_function(static function () use ($diagnosis): void { + fwrite(STDERR, $diagnosis->getMessage() . PHP_EOL); + fflush(STDERR); + flush(); + + posix_kill((int) getmypid(), SIGKILL); }); } } diff --git a/src/Runtime.php b/src/Runtime.php index d1f90d6..3a01cb0 100644 --- a/src/Runtime.php +++ b/src/Runtime.php @@ -13,6 +13,7 @@ namespace Lisachenko\NativePhpCoroutines; use Lisachenko\NativePhpCoroutines\Exception\NotShareableValueException; +use Lisachenko\NativePhpCoroutines\Exception\UndrainableCoroutineException; use Lisachenko\NativePhpCoroutines\Parallel\ArenaTaskDirectory; use Lisachenko\NativePhpCoroutines\Parallel\JoinHandle; use Lisachenko\NativePhpCoroutines\Parallel\SharedArena; @@ -263,7 +264,16 @@ public function attachResult(int $slotId): JoinHandleInterface * it loses are the ones that are bugs from inside a run anyway — a nested `run()`, a shared * root declared where only one process would see it. * + * A coroutine that was preempted and then never reached a safe point is the one thing this call + * cannot simply discard: its fiber is suspended inside the interrupt callback, where it may not + * be released. The drain is bounded, so the run ends rather than hanging, and what it could not + * drain comes back as {@see UndrainableCoroutineException} naming the coroutine and the line + * that spawned it. A panic keeps precedence — it is the bug the program actually has — and the + * straggler is reported on STDERR by the preemptor's shutdown handler in that case, which also + * ends the process before the engine can destroy the fiber. + * * @param \Closure(TaskRuntime): mixed $main + * @throws UndrainableCoroutineException */ public function run(\Closure $main): void { @@ -285,6 +295,14 @@ public function run(\Closure $main): void $this->preemptor?->disarm(); $this->supervisor?->shutdown(); } + + // Reached only when the run itself did not throw: a panic is the more informative failure + // and keeps the caller's attention, and the straggler is reported at shutdown either way. + $stragglers = $this->scheduler->undrainableCoroutines(); + + if ($stragglers !== []) { + throw new UndrainableCoroutineException($stragglers); + } } public function scheduler(): SchedulerInterface diff --git a/src/Scheduler.php b/src/Scheduler.php index 434f847..417b823 100644 --- a/src/Scheduler.php +++ b/src/Scheduler.php @@ -37,6 +37,34 @@ */ final class Scheduler implements SchedulerInterface { + /** + * Wall clock one drain attempt may spend before it starts reporting stragglers. + * + * A ceiling on resumes alone cannot bound the wait, because a single resume is not itself + * time-bounded: preemption happens between opcodes, and one `sort()` over four million integers + * defers the next slice by around two seconds (spike S4). So the drain carries a clock as well. + */ + public const float DEFAULT_DRAIN_BUDGET_SECONDS = 1.0; + + /** + * Resumes one coroutine may be given, per drain attempt, before it is reported. + * + * A coroutine that is going to cooperate does it on the first resume, or within the handful it + * takes to finish the loop it was interrupted in — the drain of a 2M-iteration loop at a 2 ms + * slice takes six (spike S7's control run). Sixty-four is an order of magnitude above that, and + * it is the floor of the budget as well as its ceiling: **every coroutine is resumed at least + * once**, so a machine so loaded that the wall clock expires before anything ran cannot produce + * a straggler report about a coroutine that was never given a chance. + * + * A coroutine that needs more CPU than this to reach its *first* safe point is reported too, + * and that is deliberate: from here there is nothing to tell it apart from one that will never + * get there, and the answer to both is the same line of code — give it a safe point. + */ + public const int DEFAULT_DRAIN_RESUMES = 64; + + /** {@see TimerQueue::now()} counts in nanoseconds; the drain budget is stated in seconds. */ + private const int NANOSECONDS_PER_SECOND = 1_000_000_000; + /** The scheduler the static surfaces (`Coroutine::spawn()`, `Io::…`, `Timer::…`) talk to. */ private static ?self $active = null; @@ -66,6 +94,29 @@ final class Scheduler implements SchedulerInterface */ private array $preemptSuspended = []; + /** + * Every coroutine that spent a whole drain budget without leaving the callback, by id. + * + * The same ownership set as {@see self::$preemptSuspended}, for the coroutines the drain has + * given up on. They are held here **forever**: the drain stopping is a decision about how long + * to keep resuming, never a decision to let go — letting go is the fatal error. What ends the + * process is {@see Preemptor} terminating it deliberately, not the engine reaching these. + * + * @var array + */ + private array $undrainable = []; + + /** + * What each of those coroutines is, and how much of the budget it consumed, by id. + * + * Kept alongside rather than derived, because the counts accumulate across drain attempts: a + * run tears down through several of them, and the report should say what the coroutine cost in + * total rather than what the last attempt happened to spend on it. + * + * @var array + */ + private array $undrainableDiagnostics = []; + private ?Preemptor $preemptor = null; private ?Coroutine $current = null; @@ -174,7 +225,8 @@ public function preemptor(): ?Preemptor } /** - * Resume every coroutine that is parked inside the preemption callback until none is left. + * Resume every coroutine parked inside the preemption callback until it leaves, or the budget + * runs out. * * A preempted fiber cannot be disposed of: its saved stack contains the FFI trampoline of the * interrupt callback, and the engine unwinds a dying fiber from wherever it is suspended, which @@ -187,30 +239,92 @@ public function preemptor(): ?Preemptor * here may be in a loop that never yields, and it is the live slice timer that guarantees the * resume returns at all rather than running to the end of that loop. * - * A coroutine that neither terminates nor ever reaches a cooperative suspension point cannot be - * discarded — this drains it forever. That is not a defect of the drain: with preemption armed, - * such a coroutine's lifetime genuinely belongs to the scheduler, and the alternative to - * spinning here is a fatal error at shutdown. + * # The budget, and what happens when it runs out * + * A coroutine with no cooperative point at all — `while (true) { $x++; }` — never leaves the + * callback, so an unbounded drain is an unexplained hang at shutdown. Each attempt therefore + * spends at most $budgetSeconds of wall clock over the whole set and at most $maxResumes + * resumes on any one coroutine, and every coroutine is resumed at least once whatever the clock + * says. + * + * Nothing is released when the budget runs out. The coroutine moves to a set this scheduler + * holds for the rest of the process ({@see self::undrainableCoroutines()} reports it), and + * {@see Preemptor} turns that report into a diagnosis and a deliberate process exit — the one + * ending that keeps the engine from ever destroying the fiber. A later attempt with its own + * budget picks the coroutine up again: giving up is about this attempt, not about the coroutine. + * + * @param float|null $budgetSeconds Wall clock for this whole attempt; null takes + * {@see self::DEFAULT_DRAIN_BUDGET_SECONDS}. + * @param int|null $maxResumes Resumes for any one coroutine; null takes + * {@see self::DEFAULT_DRAIN_RESUMES}. * @return int How many coroutines were drained out of the callback. */ - public function drainPreempted(): int + public function drainPreempted(?float $budgetSeconds = null, ?int $maxResumes = null): int { - $drained = 0; + // A resume only comes back because the next slice tick takes the CPU away again, not + // because the coroutine hands it over: with the timer down, one `step()` into a coroutine + // that never yields never returns, and the budget below is never even consulted. Nothing is + // lost by declining — anything still in the callback at this point has already been through + // a full budget with the timer live. + if ($this->preemptor?->isArmed() === false) { + return 0; + } - while ($this->preemptSuspended !== []) { - foreach ($this->preemptSuspended as $id => $coroutine) { - unset($this->preemptSuspended[$id]); + $budget = ($budgetSeconds ?? self::DEFAULT_DRAIN_BUDGET_SECONDS) * self::NANOSECONDS_PER_SECOND; + $deadline = TimerQueue::now() + (int) $budget; + $ceiling = $maxResumes ?? self::DEFAULT_DRAIN_RESUMES; + $drained = 0; - if ($this->drainOne($coroutine)) { - $drained++; - } + // Everything an earlier attempt gave up on is a candidate again: this call brings its own + // budget, and only a coroutine that refuses it again stays in the report. + $candidates = $this->preemptSuspended + $this->undrainable; + $spent = $this->undrainableDiagnostics; + + $this->preemptSuspended = []; + $this->undrainable = []; + $this->undrainableDiagnostics = []; + + foreach ($candidates as $id => $coroutine) { + if (!$coroutine->isPreemptSuspended()) { + continue; + } + + [$left, $resumes, $seconds] = $this->resumeOutOfTheCallback($coroutine, $deadline, $ceiling); + + if ($left) { + $drained++; + + continue; } + + $this->undrainable[$id] = $coroutine; + $this->undrainableDiagnostics[$id] = [ + 'id' => $id, + 'origin' => $coroutine->spawnLocation(), + 'resumes' => ($spent[$id]['resumes'] ?? 0) + $resumes, + 'seconds' => ($spent[$id]['seconds'] ?? 0.0) + $seconds, + ]; } return $drained; } + /** + * The coroutines the drain has given up on, and what each of them cost. + * + * Empty in every run where preemption is off, or where every preempted coroutine eventually + * returned or parked — which is every correct program. A non-empty answer is a diagnosis + * waiting to be raised, and the shape matches {@see self::blockedCoroutines()} on purpose: + * both are dumps keyed to a spawn site, because that is the line the reader has to go and look + * at. + * + * @return list + */ + public function undrainableCoroutines(): array + { + return array_values($this->undrainableDiagnostics); + } + /** The pending deadlines; the timer surface and `sleep()` arm their entries here. */ public function timers(): TimerQueue { @@ -277,7 +391,8 @@ public function runUntil(CoroutineInterface $coroutine): void public function discardPending(): void { // Everything below drops references, and a preempt-suspended coroutine is the one kind of - // debris that may not simply be dropped. + // debris that may not simply be dropped. Whatever the drain could not get out of the + // callback stays owned by $this->undrainable, so clearing $this->live below is still safe. $this->drainPreempted(); while (!$this->runQueue->isEmpty()) { @@ -435,22 +550,31 @@ private function trackPreemption(Coroutine $coroutine, ?SuspendCommand $command) return; } - unset($this->preemptSuspended[$coroutine->id()]); + // Out of the callback under its own steam, which also clears any earlier verdict about it: + // a coroutine that has just parked or yielded is plainly not undrainable. + unset( + $this->preemptSuspended[$coroutine->id()], + $this->undrainable[$coroutine->id()], + $this->undrainableDiagnostics[$coroutine->id()], + ); } /** - * Resume one coroutine until it is out of the preemption callback. + * Resume one coroutine until it is out of the preemption callback, or its budget is gone. * * "Out" means terminated, or suspended at a point the coroutine's own code chose — a channel - * park, a sleep, a yield. Both are safe to hold or to drop. + * park, a sleep, a yield. Both are safe to hold or to drop. The budget is checked *after* a + * resume, never before: a coroutine that has not been resumed at all has not refused anything. * - * @return bool Whether the coroutine was in the callback to begin with. + * @param int $deadline `hrtime(true)` mark shared with the rest of this drain attempt. + * @param int $maxResumes Resumes this coroutine may have. + * @return array{0: bool, 1: int, 2: float} Whether it left the callback, resumes spent, seconds + * spent. */ - private function drainOne(Coroutine $coroutine): bool + private function resumeOutOfTheCallback(Coroutine $coroutine, int $deadline, int $maxResumes): array { - if (!$coroutine->isPreemptSuspended()) { - return false; - } + $started = TimerQueue::now(); + $resumes = 0; while ($coroutine->isPreemptSuspended()) { $this->current = $coroutine; @@ -466,16 +590,22 @@ private function drainOne(Coroutine $coroutine): bool $this->current = null; } + $resumes++; + if ($command === null) { unset($this->live[$coroutine->id()]); break; } + + if ($resumes >= $maxResumes || TimerQueue::now() >= $deadline) { + break; + } } - unset($this->preemptSuspended[$coroutine->id()]); + $elapsed = (TimerQueue::now() - $started) / self::NANOSECONDS_PER_SECOND; - return true; + return [!$coroutine->isPreemptSuspended(), $resumes, $elapsed]; } /** diff --git a/tests/Functional/testACoroutineThatNeverCooperatesEndsTheRunWithADiagnosis.phpt b/tests/Functional/testACoroutineThatNeverCooperatesEndsTheRunWithADiagnosis.phpt new file mode 100644 index 0000000..9e71641 --- /dev/null +++ b/tests/Functional/testACoroutineThatNeverCooperatesEndsTheRunWithADiagnosis.phpt @@ -0,0 +1,43 @@ +--TEST-- +A coroutine that never parks and never returns ends the run with a diagnosis instead of hanging it +--INI-- +ffi.enable=1 +opcache.jit=off +error_reporting=E_ALL & ~E_DEPRECATED +--FILE-- + +--EXPECT-- +the runaway process ended on its own: yes +run() came back with the typed diagnosis: yes +the diagnosis names the coroutine and the line that spawned it: yes +and it names the remedy: yes diff --git a/tests/Functional/testAnUndrainableCoroutineIsNeverLeftForTheEngineToDestroy.phpt b/tests/Functional/testAnUndrainableCoroutineIsNeverLeftForTheEngineToDestroy.phpt new file mode 100644 index 0000000..c3265ec --- /dev/null +++ b/tests/Functional/testAnUndrainableCoroutineIsNeverLeftForTheEngineToDestroy.phpt @@ -0,0 +1,38 @@ +--TEST-- +The process holding an undrainable coroutine is ended deliberately, never by the engine destroying its fiber +--INI-- +ffi.enable=1 +opcache.jit=off +error_reporting=E_ALL & ~E_DEPRECATED +--FILE-- + +--EXPECT-- +the fiber was never destroyed by the engine: yes +the process was ended deliberately: yes +the diagnosis reached stderr before it: yes +a shutdown function registered before the runtime still ran: yes diff --git a/tests/Support/childProcess.php b/tests/Support/childProcess.php new file mode 100644 index 0000000..266ed21 --- /dev/null +++ b/tests/Support/childProcess.php @@ -0,0 +1,104 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + */ + +/** + * Running a whole PHP process from a test, with a deadline on it. + * + * Some behaviour can only be observed at the end of a process: how it exits, what it wrote on its + * way out, whether it exits at all. A test cannot observe that about itself, and a test that + * *becomes* the runaway process it is checking hangs the suite instead of failing it. So the risky + * script runs as a child here, supervised: read both pipes, hold a deadline, and `SIGKILL` it if the + * deadline passes — a regression then comes back as `timedOut: true`, which is an assertion, not a + * hang. + */ +declare(strict_types=1); + +namespace Lisachenko\NativePhpCoroutines\Tests\Support; + +/** + * Run $script in its own PHP process and wait for it, but never longer than $timeout. + * + * The child gets the same three INI settings every `.phpt` in this suite declares: they are not + * inherited from the parent's command line, and a deprecation from a dependency would otherwise + * land in the output the test is asserting on. + * + * @param string $script Absolute path of the PHP file to run. + * @param float $timeout Seconds to wait before killing it. + * @return array{stdout: string, stderr: string, signal: int, exitCode: int|null, timedOut: bool, + * seconds: float} + */ +function superviseChildProcess(string $script, float $timeout): array +{ + // The array form execs the binary directly. A string would go through a shell, which turns a + // signalled death into an ordinary exit code and writes its own "Killed" into stderr — exactly + // the two things a test about how a process ends must be able to tell apart. + $command = [ + PHP_BINARY, + '-d', 'ffi.enable=1', + '-d', 'opcache.jit=off', + '-d', 'error_reporting=E_ALL & ~E_DEPRECATED', + $script, + ]; + + $descriptors = [1 => ['pipe', 'w'], 2 => ['pipe', 'w']]; + $started = hrtime(true); + $process = proc_open($command, $descriptors, $pipes); + + if (!is_resource($process)) { + return ['stdout' => '', 'stderr' => 'proc_open() failed', 'signal' => 0, 'exitCode' => null, + 'timedOut' => false, 'seconds' => 0.0]; + } + + stream_set_blocking($pipes[1], false); + stream_set_blocking($pipes[2], false); + + $stdout = ''; + $stderr = ''; + $deadline = microtime(true) + $timeout; + $timedOut = false; + $status = proc_get_status($process); + + while ($status['running']) { + if (microtime(true) >= $deadline) { + $timedOut = true; + proc_terminate($process, SIGKILL); + + break; + } + + $read = [$pipes[1], $pipes[2]]; + $write = []; + $except = []; + + if (@stream_select($read, $write, $except, 0, 50_000) > 0) { + $stdout .= (string) stream_get_contents($pipes[1]); + $stderr .= (string) stream_get_contents($pipes[2]); + } + + $status = proc_get_status($process); + } + + $stdout .= (string) stream_get_contents($pipes[1]); + $stderr .= (string) stream_get_contents($pipes[2]); + + fclose($pipes[1]); + fclose($pipes[2]); + proc_close($process); + + return [ + 'stdout' => $stdout, + 'stderr' => $stderr, + 'signal' => $status['signaled'] ? $status['termsig'] : 0, + 'exitCode' => $status['signaled'] ? null : $status['exitcode'], + 'timedOut' => $timedOut, + 'seconds' => (hrtime(true) - $started) / 1e9, + ]; +} diff --git a/tests/Support/runawayCoroutine.php b/tests/Support/runawayCoroutine.php new file mode 100644 index 0000000..68879c7 --- /dev/null +++ b/tests/Support/runawayCoroutine.php @@ -0,0 +1,71 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + */ + +/** + * The coroutine from issue #18, in a process of its own. + * + * `while (true) { $x++; }` never returns and never parks, so the scheduler can never resume it out + * of the preemption callback. This script is not a test: it is the process whose *ending* two tests + * observe from the outside ({@see superviseChildProcess()}), because the ending is the behaviour — + * it must be prompt, it must say which coroutine and which line, and it must not be the engine + * destroying a suspended fiber. + * + * Every line it prints is an observation the supervising test greps for. + */ +declare(strict_types=1); + +use Lisachenko\NativePhpCoroutines\Coroutine; +use Lisachenko\NativePhpCoroutines\Exception\UndrainableCoroutineException; +use Lisachenko\NativePhpCoroutines\Runtime; + +require __DIR__ . '/../../vendor/autoload.php'; + +// Registered before the runtime arms preemption, so the runtime's own shutdown handler is queued +// after this one: if the diagnosis were to terminate the process the moment it is produced, this +// line would go missing. +register_shutdown_function(static function (): void { + echo 'CHILD: a shutdown function registered before the runtime still ran', PHP_EOL; +}); + +$runtime = new Runtime(preemptive: true); + +try { + $runtime->run(static function (): void { + Coroutine::spawn(static function (): void { + $x = 0; + + while (true) { + $x++; + } + }); + + // Hand the CPU over once, so the runaway is running when main returns. From there Go + // semantics discard everything still pending — except this one, which cannot be. + Coroutine::yield(); + }); + + echo 'CHILD: run() returned with no diagnosis', PHP_EOL; +} catch (UndrainableCoroutineException $diagnosis) { + echo 'CHILD: run() threw ', $diagnosis::class, PHP_EOL; + + foreach ($diagnosis->stragglers() as $straggler) { + printf( + "CHILD: straggler #%d spawned at %s after %d resume(s)\n", + $straggler['id'], + $straggler['origin'], + $straggler['resumes'], + ); + } + + echo $diagnosis->getMessage(), PHP_EOL; +} + +echo 'CHILD: reached the end of the script', PHP_EOL;