diff --git a/docs/long-running.md b/docs/long-running.md index be790bfc..c5563610 100644 --- a/docs/long-running.md +++ b/docs/long-running.md @@ -74,6 +74,11 @@ structures (class entries, handler blocks) never carry a pointer into the next r freed trampolines. After shutdown, z-engine performs no engine writes at all — hooks are inactive during shutdown-phase object destructors, and installing a new hook throws. +`ObserverHook` (the `zend_observer` fcall bridge) follows this same lifecycle but has an extra +boot-time constraint: it can only be installed from the `Core::preload()` path and only when a +startup-time observer provider has enabled the engine observer machinery, otherwise it refuses with +a typed exception rather than corrupting memory. See [observer-hook.md](observer-hook.md). + ### Runtime models - **Worker loops** (RoadRunner, Swoole, ReactPHP, FrankenPHP worker mode): the whole worker diff --git a/docs/observer-hook.md b/docs/observer-hook.md new file mode 100644 index 00000000..a51b32e0 --- /dev/null +++ b/docs/observer-hook.md @@ -0,0 +1,189 @@ +# Observer hook (`zend_observer` fcall bridge) + +`ObserverHook` bridges the engine's `zend_observer` fcall *begin*/*end* handlers to userland +callbacks, following the same install/uninstall/reinstall lifecycle and `Core` hook registry +semantics as the other hooks (see [long-running.md](long-running.md#hook-lifecycle)). It targets a +single `zend_function` and attaches a begin and an end handler through the engine's per-function +runtime API (`zend_observer_add_begin_handler` / `zend_observer_add_end_handler`). + +```php +// From the opcache.preload script only (see below): +$function = (new ReflectionFunction('some_function'))->getRawFunctionPointer(); +Core::observeFunction( + $function, + fn(ExecutionData $frame) => /* begin */, + fn(ExecutionData $frame, ?ReflectionValue $return) => /* end; omit for a begin-only hook */, +); +``` + +The callbacks receive an [`ExecutionData`](../src/System/ExecutionData.php) frame; the end callback +also receives the return value (or `null` for abrupt/generator returns). Exceptions thrown by a +callback are contained and downgraded to an `E_USER_WARNING` — a throw must never cross the FFI +boundary into the engine ([#50](https://github.com/lisachenko/z-engine/issues/50)). + +The full firing path — begin/end with return values, clean uninstall, nested-call ordering, +throwing functions under begin-only hooks, containment, and internal functions — is verified +end-to-end by [`ObserverHookFiringTest`](../tests/System/Hook/ObserverHookFiringTest.php) against +the reference provider described below. + +## The hard constraints + +The `zend_observer` fcall machinery is designed for **C extensions that register during MINIT**, and +that assumption leaks into every part of its API. The constraints below are not policy choices; +they are what the engine does, verified against php-src 8.4.19. + +### 1. Registration timing — preload only, and even that is too late to *enable* observers + +`zend_observer_fcall_register()` is only honoured before startup finishes. The engine freezes the +observer configuration in `zend_observer_post_startup()`, which runs at the tail of +`php_module_startup()` (`main.c`), **before** the `opcache.preload` script executes — preloading is +driven from `zend_post_startup()` → `accel_post_startup()` → `accel_finish_startup()`, which the +engine calls *after* `zend_observer_post_startup()`. + +Consequently, by the time `Core::preload()` runs, `zend_observer_fcall_op_array_extension` is already +`-1` (observers disabled) unless a startup-time provider reserved the slot. This is directly +observable: + +``` +$ php -d ffi.enable=1 -d opcache.enable_cli=1 \ + -d opcache.preload=probe.php -r '' +# probe.php, during preload: +op_array_extension = -1 # ZEND_OBSERVER_ENABLED is false +``` + +`ObserverHook` therefore requires the preload boot path (`Core::isPreloaded()`), and refuses with +`ObserverException::notPreloaded()` under a plain `Core::init()` request. But the preload requirement +is necessary, not sufficient: the machinery must additionally have been *enabled* by a startup +provider (next point). + +### 2. Already-compiled op_arrays and the retroactive-stamping verdict + +Observer support is stamped into each function at **compile time**, in `pass_two` +(`zend_opcode.c`): `op_array->cache_size = zend_observer_fcall_op_array_extension_handles * +sizeof(void*)` is set in `init_op_array`, and `op_array->T += ZEND_OBSERVER_ENABLED` reserves the +per-frame temporary that stores the observed-frame linked list. Internal functions get one shared +`run_time_cache` block sized once at startup by `zend_init_internal_run_time_cache()`. + +This makes **retroactive stamping unsafe** and, in fact, makes userland self-enablement impossible: + +- A function compiled while observers were **disabled** has a `cache_size` and `T` that do **not** + include an observer slot. Writing observer handler data into its `run_time_cache`, or enabling + observers so the VM reads a `prev_observed_frame` temporary the frame never reserved, is an + out-of-bounds access — heap and stack corruption. +- Internal functions share a single startup-sized cache block; growing the extension handle count + afterwards cannot grow that block. + +Enabling observers late (setting `zend_observer_fcall_op_array_extension` by hand from the preload +script) was tested and **segfaults**: the engine's observer install path invokes the registered +`zend_observer_fcall_init` — a callback that returns a struct by value — from inside call-frame +setup, and driving that through an FFI trampoline corrupts execution state +(`SIGSEGV` on the first observed call). z-engine therefore never self-enables observers. + +**Observed/unobserved boundary.** Only functions compiled **after** the engine's observer machinery +was enabled — by a startup-time provider — can be observed. On a stock z-engine build with no such +provider, observers are disabled and `ObserverHook::install()` refuses with +`ObserverException::observersDisabled()` rather than corrupting memory. This boundary is pinned by a +test: [`ObserverHookPreloadTest`](../tests/System/Hook/ObserverHookPreloadTest.php) boots through the +preload path and asserts `PRELOADED=1`, `OBSERVER_ENABLED=0`, `OBSERVE=rejected`. + +### 3. Callback-exception containment — and why throwing functions need begin-only hooks + +`handleBegin()` / `handleEnd()` wrap the userland callback in a catch-all that downgrades any +`Throwable` to an `E_USER_WARNING` (a user error handler converting that warning back into an +exception is swallowed too), exactly like the other FFI-callback hooks +([#50](https://github.com/lisachenko/z-engine/issues/50)). This is verified end-to-end: a begin +callback that throws produces the warning and the function call — including its end handler — +continues unharmed. + +There is a second, harder containment problem that **cannot** be solved from userland: the engine +invokes **end handlers while unwinding a throwing frame**, i.e. with `EG(exception)` set — and +ext/ffi refuses to run any callback in that state. `zend_call_function()` skips the PHP closure +outright when `EG(exception)` is set ("we would result in an unstable executor otherwise"), and the +FFI trampoline then aborts the whole process with the fatal error *"Throwing from FFI callbacks is +not allowed"* — all in C, before any z-engine code gets control. Therefore: + +> **A function that can throw must be observed with a begin-only hook** (`$end = null` / +> omitted). Begin handlers run at frame entry, where no exception can be in flight, and the +> exception then propagates through the observed function exactly as without the hook. + +Both sides are pinned by [`ObserverHookFiringTest`](../tests/System/Hook/ObserverHookFiringTest.php): +the begin-only hook observes the throwing function and the exception is caught normally, while a +deliberately attached end handler reproduces the documented ext/ffi abort in a sacrificial child +process. If a future PHP release lifts the ext/ffi restriction, that pin fails and the begin-only +rule can be revisited. + +### 4. Internal vs userland functions + +For a **user function**, `install()` warms the lazily-allocated `run_time_cache` +(`zend_init_func_run_time_cache`) so the observer slot exists before the first call, then attaches +via the op_array observer extension slot. For an **internal function**, observation uses the separate +`zend_observer_fcall_internal_function_extension` slot and the startup-sized shared cache block; +z-engine refuses whenever that slot is `-1`, because the block is frozen at startup and cannot be +grown from userland. Both kinds fire verifiably +([`ObserverHookFiringTest`](../tests/System/Hook/ObserverHookFiringTest.php) asserts begin/end for a +preload-compiled user function and for `strrev`), and the guard paths are covered by +[`ObserverHookTest`](../tests/System/Hook/ObserverHookTest.php) / +[`ObserverHookPreloadTest`](../tests/System/Hook/ObserverHookPreloadTest.php). + +Note on `zend_execute_internal`-based paths: observer begin/end for internal functions is driven by +the *calling* op_array's `DO_ICALL`/`DO_FCALL` observer handler variants, not by replacing +`zend_execute_internal`, so the two interception mechanisms are independent and can coexist. + +### 5. JIT + +Out of scope — z-engine already requires `opcache.jit=off`. + +## The reference startup-time provider + +z-engine ships the minimal provider as a test fixture: +[`tests/fixtures/observer-enabler`](../tests/fixtures/observer-enabler/observer_enabler.c) — a +~50-line extension whose MINIT registers an fcall observer returning `{NULL, NULL}` handlers for +every function. Registering it is enough to make the engine reserve the observer extension slots +(`ZEND_OBSERVER_ENABLED` becomes true) while observing nothing itself; the per-function runtime API +then becomes fully usable by `ObserverHook`. Consumers who want observer support in production can +replicate it verbatim (build with `phpize && ./configure && make`, load with `extension=...`), or +load any existing observer-registering extension instead. +[`ObserverHookFiringTest`](../tests/System/Hook/ObserverHookFiringTest.php) builds this fixture on +demand with the local toolchain and skips cleanly when `phpize`/`cc` are unavailable. + +## Slot priming and provider interaction + +The engine initialises a function's observer handler slots lazily, on the function's first call in +a request, by walking every registered provider's init callback (`zend_observer_fcall_install`); the +runtime add-handler API is only legal on initialised slots. `ObserverHook::install()` therefore +primes a never-called function's slots itself, writing the engine's own `NOT_OBSERVED` sentinel — +exactly what the install routine would write for a `{NULL, NULL}` provider — before attaching. + +Two consequences, both accepted and documented: + +- Priming marks the function "installed", so **other providers' lazy init callbacks are not + consulted for that function** for the rest of the request. With the reference enabler (which + observes nothing) this changes nothing; alongside a real observing extension it means a + z-engine-hooked function is not seen by that extension's per-function init in the same request. +- The engine reserves exactly `2 × count` handler slots per function (count = registered + providers, derived via `Core::observerFcallObserverCount()`), and z-engine cannot prove a second + begin/end pair would fit — so **only one `ObserverHook` per function** is allowed; + a second `install()` throws `ObserverException::alreadyObserved()`. + +## Lifecycle and long-running processes + +`ObserverHook` registers in the `Core` hook registry under the synthetic key +`observer-fcall::`, so `Core::shutdown()` detaches every still-installed hook while +the libffi trampolines are guaranteed alive (`zend_observer_remove_begin_handler` / +`remove_end_handler`), and `Core::reinstallHooks()` re-mints the begin/end trampolines for SAPIs that +cycle FFI callback state between requests. Each installed hook holds up to two live trampolines +(begin, and end when attached); both are owned by ext/ffi and freed at its `RSHUTDOWN`, covered by +the generic "one live libffi trampoline per installed hook" row in the +[immortal allocation table](long-running.md). + +## Summary + +| Requirement | Behaviour | +|-------------|-----------| +| Non-preload boot (`Core::init()`) | `ObserverException::notPreloaded()` | +| Preload boot, observers disabled (stock build) | `ObserverException::observersDisabled()` | +| Preload boot, observers enabled by a startup provider | begin/end fire for functions compiled after enablement, userland and internal (verified) | +| Second hook on the same function | `ObserverException::alreadyObserved()` | +| Callback throws | contained, `E_USER_WARNING`, execution continues | +| Observed function throws | supported with a begin-only hook; an end handler would be aborted by ext/ffi (pinned) | +| `Core::shutdown()` | handlers detached while trampolines are alive | diff --git a/include/8.4/linux-x64-nts/engine.h b/include/8.4/linux-x64-nts/engine.h index f9e93c2a..43874056 100644 --- a/include/8.4/linux-x64-nts/engine.h +++ b/include/8.4/linux-x64-nts/engine.h @@ -386,6 +386,8 @@ typedef struct _zend_lazy_objects_store { HashTable infos; } zend_lazy_objects_store; typedef struct _zend_property_info zend_property_info; +typedef struct _zend_fcall_info zend_fcall_info; +typedef struct _zend_fcall_info_cache zend_fcall_info_cache; struct _zend_property_info; typedef zval *(*zend_object_read_property_t)(zend_object *object, zend_string *member, int type, void **cache_slot, zval *rv); typedef zval *(*zend_object_read_dimension_t)(zend_object *object, zval *offset, int type, zval *rv); @@ -882,6 +884,22 @@ struct _zend_function_entry { const zend_frameless_function_info *frameless_function_infos; const char *doc_comment; }; +struct _zend_fcall_info { + size_t size; + zval function_name; + zval *retval; + zval *params; + zend_object *object; + uint32_t param_count; + HashTable *named_params; +}; +struct _zend_fcall_info_cache { + zend_function *function_handler; + zend_class_entry *calling_scope; + zend_class_entry *called_scope; + zend_object *object; + zend_object *closure; +}; struct _zend_ini_entry { zend_string *name; int (*on_modify)(zend_ini_entry *entry, zend_string *new_value, void *mh_arg1, void *mh_arg2, void *mh_arg3, int stage); @@ -946,6 +964,50 @@ typedef struct _zend_lex_state { zend_ast *ast; zend_arena *ast_arena; } zend_lex_state; +typedef enum { + ZEND_FIBER_STATUS_INIT, + ZEND_FIBER_STATUS_RUNNING, + ZEND_FIBER_STATUS_SUSPENDED, + ZEND_FIBER_STATUS_DEAD, +} zend_fiber_status; +typedef struct _zend_fiber_stack zend_fiber_stack; +typedef struct _zend_fiber_transfer { + zend_fiber_context *context; + zval value; + uint8_t flags; +} zend_fiber_transfer; +typedef void (*zend_fiber_coroutine)(zend_fiber_transfer *transfer); +typedef void (*zend_fiber_clean)(zend_fiber_context *context); +struct _zend_fiber_context { + void *handle; + void *kind; + zend_fiber_coroutine function; + zend_fiber_clean cleanup; + zend_fiber_stack *stack; + zend_fiber_status status; + zend_execute_data *top_observed_frame; + void *reserved[6]; +}; +struct _zend_fiber { + zend_object std; + uint8_t flags; + zend_fiber_context context; + zend_fiber_context *caller; + zend_fiber_context *previous; + zend_fcall_info fci; + zend_fcall_info_cache fci_cache; + zend_execute_data *execute_data; + zend_execute_data *stack_bottom; + zend_vm_stack vm_stack; + zval result; +}; +typedef void (*zend_observer_fcall_begin_handler)(zend_execute_data *execute_data); +typedef void (*zend_observer_fcall_end_handler)(zend_execute_data *execute_data, zval *retval); +typedef struct _zend_observer_fcall_handlers { + zend_observer_fcall_begin_handler begin; + zend_observer_fcall_end_handler end; +} zend_observer_fcall_handlers; +typedef zend_observer_fcall_handlers (*zend_observer_fcall_init)(zend_execute_data *execute_data); typedef struct _zend_closure { zend_object std; zend_function func; @@ -964,6 +1026,13 @@ extern zval * zend_hash_index_find(const HashTable *, zend_ulong); extern void zend_hash_destroy(HashTable *); extern zend_result zend_set_user_opcode_handler(uint8_t, user_opcode_handler_t); extern user_opcode_handler_t zend_get_user_opcode_handler(uint8_t); +extern void zend_observer_fcall_register(zend_observer_fcall_init); +extern void zend_observer_add_begin_handler(zend_function *, zend_observer_fcall_begin_handler); +extern void zend_observer_add_end_handler(zend_function *, zend_observer_fcall_end_handler); +extern _Bool zend_observer_remove_begin_handler(zend_function *, zend_observer_fcall_begin_handler, zend_observer_fcall_begin_handler *); +extern _Bool zend_observer_remove_end_handler(zend_function *, zend_observer_fcall_end_handler, zend_observer_fcall_end_handler *); +extern void zend_init_func_run_time_cache(zend_op_array *); +extern size_t zend_internal_run_time_cache_reserved_size(void); extern void zend_do_inheritance_ex(zend_class_entry *, zend_class_entry *, _Bool); extern zend_object * zend_objects_new(zend_class_entry *); extern void zend_object_std_init(zend_object *, zend_class_entry *); @@ -1003,3 +1072,5 @@ extern struct _zend_compiler_globals compiler_globals; extern HashTable module_registry; extern const zend_object_handlers std_object_handlers; extern zend_ast_process_t zend_ast_process; +extern int zend_observer_fcall_op_array_extension; +extern int zend_observer_fcall_internal_function_extension; diff --git a/src/Core.php b/src/Core.php index 07315da4..2eac84cc 100644 --- a/src/Core.php +++ b/src/Core.php @@ -24,6 +24,7 @@ use ZEngine\System\Compiler; use ZEngine\System\Executor; use ZEngine\System\Hook\AstProcessHook; +use ZEngine\System\Hook\ObserverHook; use ZEngine\Type\HashTable; /** @@ -244,6 +245,16 @@ class Core */ private static bool $shutdownRegistered = false; + /** + * Whether Core::preload() was the boot path for this process. + * + * The engine freezes its fcall-observer configuration during startup + * (zend_observer_post_startup), before any request begins, so features that + * require the observer machinery are only meaningful when z-engine booted + * from the opcache.preload script. See src/System/Hook/ObserverHook.php. + */ + private static bool $isPreloaded = false; + /** * Performs Z-engine core initialization */ @@ -300,6 +311,114 @@ public static function preload(): void // Performs initialization of properties, otherwise we will get an error about uninitialized properties Core::init(); + + // Record that observer registration timing is available for this process + self::$isPreloaded = true; + } + + /** + * Checks whether z-engine booted through Core::preload() (the opcache.preload path) + * + * @see ZEngine\System\Hook\ObserverHook for why the observer bridge requires it + */ + public static function isPreloaded(): bool + { + return self::$isPreloaded; + } + + /** + * Checks whether the engine fcall-observer machinery is enabled (ZEND_OBSERVER_ENABLED) + * + * The extension slot globals are -1 until a startup-time (MINIT) observer provider reserves + * them in zend_observer_post_startup(); once frozen at -1 they cannot be enabled from userland. + * + * @param bool $forUserFunction true to test the op_array (user function) slot, false for the + * internal-function slot + */ + public static function isObserverEnabled(bool $forUserFunction = true): bool + { + return self::observerFcallExtensionSlot($forUserFunction) !== -1; + } + + /** + * Returns the run_time_cache slot index reserved for observer handlers (-1 when disabled) + * + * The engine reserves one contiguous block of 2*count slots per function kind in + * zend_observer_post_startup(); this is the index of the block's first slot. + * + * @param bool $forUserFunction true for the op_array (user function) slot, false for the + * internal-function slot + * @internal used by ObserverHook to locate a function's observer handler slots + */ + public static function observerFcallExtensionSlot(bool $forUserFunction): int + { + if ($forUserFunction) { + // @phpstan-ignore property.notFound (FFI global read, dynamically typed) + $extension = self::$engine->zend_observer_fcall_op_array_extension; + } else { + // @phpstan-ignore property.notFound (FFI global read, dynamically typed) + $extension = self::$engine->zend_observer_fcall_internal_function_extension; + } + assert(is_int($extension)); + + return $extension; + } + + /** + * Returns the number of fcall observers registered with the engine at startup + * + * Derived without touching engine privates: the observer block is always the LAST + * internal-handle reservation (zend_observer_post_startup() runs after every MINIT and + * nothing may reserve handles later - the internal run_time_cache is sized immediately + * afterwards), so the block spans from the internal observer slot to the total reserved + * size, and holds exactly 2 slots (begin + end) per registered observer. + * + * @internal used by ObserverHook to compute the observer slot layout + */ + public static function observerFcallObserverCount(): int + { + $internalSlot = self::observerFcallExtensionSlot(false); + if ($internalSlot === -1) { + return 0; + } + $reservedBytes = self::call('zend_internal_run_time_cache_reserved_size'); + assert(is_int($reservedBytes)); + + return intdiv(intdiv($reservedBytes, PHP_INT_SIZE) - $internalSlot, 2); + } + + /** + * Resolves a ZEND_MAP_PTR field value to the real address it maps to (0 for NULL) + * + * A map pointer field stores either a real pointer (even value) or an odd offset that must + * be resolved through the biased CG(map_ptr_base); this mirrors ZEND_MAP_PTR_GET. + * + * @param CData|null $mapPtrField Value read from a *__ptr map pointer field (null when the + * engine stored NULL) + * @internal used by ObserverHook to reach a function's run_time_cache + */ + public static function mapPtrGet(?CData $mapPtrField): int + { + if ($mapPtrField === null) { + return 0; + } + $raw = self::addressOf($mapPtrField); + if (($raw & 1) === 0) { + return $raw; + } + + // Offset form: resolve against the biased base (the -1 bias of the base and the +1 tag + // of the offset cancel out, exactly like ZEND_MAP_PTR_OFFSET2PTR) + // @phpstan-ignore property.notFound (FFI global read, dynamically typed) + $compilerGlobals = self::$engine->compiler_globals; + assert($compilerGlobals instanceof CData); + $base = $compilerGlobals->map_ptr_base; + assert($base instanceof CData); + $slot = self::pointerAtAddress('uintptr_t *', self::addressOf($base) + $raw); + $value = $slot[0]; + assert(is_int($value)); + + return $value; } /** @@ -830,6 +949,27 @@ public static function setASTProcessHandler(Closure $handler): AstProcessHook return $hook; } + /** + * Installs an ObserverHook bridging the engine fcall begin/end observers to userland callbacks + * + * Only meaningful from the opcache.preload path and only when a startup-time observer provider + * has enabled the engine observer machinery; the hook throws a typed ObserverException + * otherwise (never a silent no-op, never a memory-unsafe write). See docs/observer-hook.md. + * + * @param CData $function zend_function* to observe (e.g. ReflectionFunction::getRawFunctionPointer()) + * @param Closure $begin function(ExecutionData $frame): void, invoked as the call is entered + * @param Closure|null $end function(ExecutionData $frame, ?ReflectionValue $return): void, on + * return; omit for a begin-only hook (REQUIRED for functions that can + * throw, see docs/observer-hook.md) + */ + public static function observeFunction(CData $function, Closure $begin, ?Closure $end = null): ObserverHook + { + $hook = new ObserverHook($function, $begin, $end); + $hook->install(); + + return $hook; + } + /** * This method preloads all framework classes to bypass all possible hooks */ diff --git a/src/Reflection/FunctionLikeTrait.php b/src/Reflection/FunctionLikeTrait.php index e5c07e0e..55857a59 100644 --- a/src/Reflection/FunctionLikeTrait.php +++ b/src/Reflection/FunctionLikeTrait.php @@ -521,6 +521,22 @@ private function ensureCompatibleClosure(\Closure $newCode): void } } + /** + * Returns a pointer to the underlying zend_function structure + * + * Engine APIs that take a `zend_function *` (for example the zend_observer + * per-function handler attachers) need the address of the reflected entry, + * not a copy of it. The entry is referenced directly, so the caller writes + * through to the live engine structure. + * + * @internal + */ + public function getRawFunctionPointer(): CData + { + // $this->pointer is already the zend_function* the reflection wraps + return $this->pointer; + } + /** * Returns a pointer to the common structure (to work natively with zend_function and zend_internal_function) */ diff --git a/src/System/Hook/ObserverException.php b/src/System/Hook/ObserverException.php new file mode 100644 index 00000000..e0db579e --- /dev/null +++ b/src/System/Hook/ObserverException.php @@ -0,0 +1,68 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace ZEngine\System\Hook; + +use RuntimeException; + +/** + * Raised when an ObserverHook cannot be installed on the current engine state. + * + * The zend_observer fcall machinery has hard timing and memory-safety + * preconditions that a userland/FFI consumer cannot satisfy on every build. + * Rather than silently doing nothing (a call that never fires) or writing into + * structures the engine sized without an observer slot (memory corruption), + * ObserverHook refuses with this typed exception. See docs/observer-hook.md for + * the full boundary description. + */ +final class ObserverException extends RuntimeException +{ + /** + * Observer registration was attempted outside the opcache.preload boot path + */ + public static function notPreloaded(): self + { + return new self( + 'ObserverHook can only be installed from the opcache.preload script (Core::preload()). ' + . 'The engine freezes its observer configuration during startup, before a normal ' + . 'Core::init() request begins, so a non-preload setup cannot attach fcall observers. ' + . 'See docs/observer-hook.md.', + ); + } + + /** + * A z-engine observer hook is already attached to the target function + */ + public static function alreadyObserved(): self + { + return new self( + 'An ObserverHook is already attached to this function. The engine reserves exactly one ' + . 'begin/end handler pair per registered observer, and z-engine cannot prove a second ' + . 'pair would fit - uninstall the existing hook first.', + ); + } + + /** + * The engine's fcall-observer machinery is not enabled on this build + */ + public static function observersDisabled(): self + { + return new self( + 'The engine fcall-observer machinery is disabled (ZEND_OBSERVER_ENABLED is false: ' + . 'zend_observer_fcall_op_array_extension == -1). z-engine cannot enable it from userland ' + . 'because zend_observer_post_startup() has already frozen it by the time the preload script ' + . 'runs, and forcing it on corrupts every op_array compiled without an observer slot. ' + . 'A startup-time (MINIT) observer provider must enable observers first. See docs/observer-hook.md.', + ); + } +} diff --git a/src/System/Hook/ObserverHook.php b/src/System/Hook/ObserverHook.php new file mode 100644 index 00000000..c4a47f09 --- /dev/null +++ b/src/System/Hook/ObserverHook.php @@ -0,0 +1,460 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace ZEngine\System\Hook; + +use Closure; +use FFI\CData; +use ZEngine\Core; +use ZEngine\Hook\HookInterface; +use ZEngine\Reflection\ReflectionValue; +use ZEngine\System\ExecutionData; + +/** + * ObserverHook bridges the engine's zend_observer fcall begin/end handlers to userland callbacks. + * + * It targets one zend_function and attaches a begin and an end handler through the engine's + * per-function runtime API (zend_observer_add_begin_handler / zend_observer_add_end_handler), + * following the same install/uninstall/reinstall lifecycle and Core hook registry semantics as + * OpCodeHook. Each installed hook lives under the synthetic registry key + * "observer-fcall::", and Core::shutdown() unwinds still-installed hooks while + * the FFI trampolines are guaranteed alive. + * + * Hard precondition. + * The zend_observer fcall machinery has to be enabled by a startup-time (MINIT) observer provider + * BEFORE the hook is installed. z-engine cannot enable it from userland: zend_observer_post_startup() + * freezes zend_observer_fcall_op_array_extension during engine startup, before the opcache.preload + * script runs, and forcing it on afterwards corrupts every op_array/internal function whose stack + * frame and run_time_cache were sized without an observer slot. install() therefore refuses with an + * ObserverException when the machinery is disabled instead of corrupting memory. Only functions + * compiled while observers were already enabled can be observed. See docs/observer-hook.md for the + * timing analysis, the retroactive-stamping verdict and the observed/unobserved boundary. + * + * Throwing functions need begin-only hooks. + * The engine invokes end handlers while unwinding a throwing frame, i.e. with EG(exception) set - + * and ext/ffi refuses to run ANY callback in that state: zend_call_function() skips the PHP + * closure and the trampoline aborts the process with "Throwing from FFI callbacks is not allowed" + * before z-engine gets control. There is no userland fix (the abort happens in C, before any PHP + * runs), so a function that can throw must be observed with a begin-only hook ($end = null); the + * limitation is pinned by ObserverHookFiringTest and documented in docs/observer-hook.md. + */ +final class ObserverHook implements HookInterface +{ + /** + * Prefix of the synthetic Core registry key: observer handlers are attached per zend_function, + * not into a hookable struct field, and every target function forms its own chain + */ + private const FIELD_KEY_PREFIX = 'observer-fcall'; + + /** + * Sentinel the engine stores in an initialised-but-unattached observer handler slot + * (ZEND_OBSERVER_NOT_OBSERVED in zend_observer.c). The runtime add-handler API requires the + * slot to hold a sentinel (or a real handler) - a NULL slot means "engine has not initialised + * observer state for this function in this request yet" and must be primed first. + */ + private const ZEND_OBSERVER_NOT_OBSERVED = 2; + + /** + * Target zend_function pointer (the observed function/method) + */ + private CData $function; + + /** + * User begin callback: function(ExecutionData $frame): void + */ + private Closure $beginHandler; + + /** + * User end callback: function(ExecutionData $frame, ?ReflectionValue $returnValue): void + * + * Null for a begin-only hook: an end handler is an FFI trampoline the engine invokes while + * unwinding a throwing frame, which ext/ffi aborts on (see the class doc), so functions that + * may throw must be observed begin-only. + */ + private ?Closure $endHandler; + + /** + * Whether the observed function is a user (op_array) function; internal functions use a + * different observer extension slot + */ + private bool $isUserFunction; + + /** + * Stable holder for the begin trampoline (kept as a struct field CData so the same function + * pointer is used for both attach and removal, and so libffi never collects it while installed) + * + * @var CData|null single-element array of zend_observer_fcall_begin_handler + */ + private ?CData $beginSlot = null; + + /** + * Stable holder for the end trampoline + * + * @var CData|null single-element array of zend_observer_fcall_end_handler + */ + private ?CData $endSlot = null; + + /** + * Whether this hook's handlers are currently attached to the target function + */ + private bool $installed = false; + + /** + * @param CData $function zend_function* the handlers are attached to (a + * zend_internal_function* is accepted and normalized) + * @param Closure $begin function(ExecutionData $frame): void + * @param Closure|null $end function(ExecutionData $frame, ?ReflectionValue $returnValue): void, + * or null for a begin-only hook (REQUIRED for functions that can + * throw: an end handler invoked during exception unwinding is + * aborted by ext/ffi, see docs/observer-hook.md) + */ + public function __construct(CData $function, Closure $begin, ?Closure $end = null) + { + // Normalize to the union type: the engine observer API takes zend_function*, while + // reflection returns zend_internal_function* for internal entries + $this->function = Core::cast('zend_function *', $function); + $this->beginHandler = $begin; + $this->endHandler = $end; + + // zend_function.type is ZEND_INTERNAL_FUNCTION (1), ZEND_USER_FUNCTION (2) or + // ZEND_EVAL_CODE (4); the is_int() guard narrows the dynamically typed CData read to int + $type = $this->function->type; + $this->isUserFunction = is_int($type) && ($type & Core::ZEND_USER_FUNCTION) !== 0; + } + + /** + * Attaches the begin/end handlers to the target function (idempotent) + * + * Refuses (typed ObserverException) rather than corrupting memory when the engine's observer + * machinery is unavailable: outside the preload boot path, or when observers were not enabled + * by a startup provider for this function kind. + */ + public function install(): void + { + if ($this->installed) { + return; + } + if (Core::isShutdown()) { + throw new \LogicException('Cannot install an engine hook after Core::shutdown()'); + } + if (!Core::isPreloaded()) { + throw ObserverException::notPreloaded(); + } + if (!Core::isObserverEnabled($this->isUserFunction)) { + throw ObserverException::observersDisabled(); + } + if (Core::topHook($this->getHookFieldKey()) !== null) { + // The engine reserves exactly 2*count handler slots per function; z-engine cannot + // prove there is room for a second begin/end pair, and overflowing the block is + // undefined behaviour in the engine (ZEND_UNREACHABLE) - refuse instead of stacking + throw ObserverException::alreadyObserved(); + } + + $this->primeObserverSlots(); + $this->attachHandlers(); + + $this->installed = true; + Core::registerHook($this); + } + + /** + * Detaches the begin/end handlers from the target function (idempotent) + * + * Only the most recently installed hook of a function may be uninstalled: removing an older + * hook first would leave a newer trampoline referenced by the engine. + */ + public function uninstall(): void + { + if (!$this->installed) { + return; + } + if (Core::isShutdown()) { + // The engine already tore down its observer state during request shutdown + $this->installed = false; + + return; + } + if (!Core::isTopHook($this)) { + throw new \LogicException( + 'Another observer hook was installed over this one on the same function; uninstall it first', + ); + } + + $this->detachHandlers(); + + $this->installed = false; + Core::unregisterHook($this); + } + + /** + * Re-installs the hook with freshly minted trampolines (uninstall + install) + */ + public function reinstall(): void + { + $this->uninstall(); + $this->install(); + } + + /** + * @inheritDoc + */ + public function isInstalled(): bool + { + return $this->installed; + } + + /** + * Observer handlers have no predecessor pointer to proceed into (z-engine attaches its own + * begin/end handlers directly), so there is never an original handler to call. + */ + public function hasOriginalHandler(): bool + { + return false; + } + + /** + * @inheritDoc + */ + public function getHookFieldKey(): string + { + return self::FIELD_KEY_PREFIX . '::' . Core::addressOf($this->function); + } + + /** + * @inheritDoc + */ + public function refreshTrampoline(): void + { + if (!$this->installed) { + return; + } + // Detach the stale trampolines and mint fresh ones (SAPIs that cycle FFI callback state) + $this->detachHandlers(); + $this->attachHandlers(); + } + + /** + * Makes sure the target function's observer handler slots exist and are initialised + * + * The engine initialises a function's observer slots lazily, on the function's first call in + * a request (zend_observer_fcall_install), and its runtime add-handler API is only legal on + * initialised slots. Two preparation steps replicate what the engine would do: + * + * 1. A user function's run_time_cache is allocated on demand (zend_init_func_run_time_cache) + * - the engine allocates it lazily on the first call anyway, and never before the slots + * are consulted. An internal function's cache lives in the single startup-sized block and + * must already exist; z-engine never grows it. + * 2. A never-called function's slots are zero: they are primed with the engine's own + * NOT_OBSERVED sentinel, exactly like zend_observer_fcall_install does before attaching + * handlers. Priming marks the function as "installed", so other observer providers' + * lazy init callbacks are not consulted for this function for the rest of the request - + * an accepted trade-off documented in docs/observer-hook.md. + * + * The slot block layout is [begin(0) .. begin(count-1), end(0) .. end(count-1)] at the + * reserved extension slot index of the function's run_time_cache, where count is the number + * of observers registered at startup (Core::observerFcallObserverCount()). + */ + private function primeObserverSlots(): void + { + $observerCount = Core::observerFcallObserverCount(); + if ($observerCount < 1) { + throw ObserverException::observersDisabled(); + } + + $runTimeCache = $this->runTimeCacheAddress(); + if ($runTimeCache === 0) { + if (!$this->isUserFunction) { + // The startup-sized internal cache block must already cover this function + throw ObserverException::observersDisabled(); + } + $opArray = $this->function->op_array; + assert($opArray instanceof CData); + Core::call('zend_init_func_run_time_cache', Core::addr($opArray)); + $runTimeCache = $this->runTimeCacheAddress(); + } + if ($runTimeCache === 0) { + throw ObserverException::observersDisabled(); + } + + $slotIndex = Core::observerFcallExtensionSlot($this->isUserFunction); + $beginSlot = Core::pointerAtAddress('uintptr_t *', $runTimeCache + $slotIndex * PHP_INT_SIZE); + if ($beginSlot[0] === 0) { + // Never called this request: initialise like zend_observer_fcall_install would + $endSlot = Core::pointerAtAddress('uintptr_t *', $runTimeCache + ($slotIndex + $observerCount) * PHP_INT_SIZE); + $beginSlot[0] = self::ZEND_OBSERVER_NOT_OBSERVED; + $endSlot[0] = self::ZEND_OBSERVER_NOT_OBSERVED; + } + } + + /** + * Resolves the target function's run_time_cache base address (0 when not allocated yet) + */ + private function runTimeCacheAddress(): int + { + $common = $this->function->common; + assert($common instanceof CData); + $mapPtrField = $common->run_time_cache__ptr; + assert($mapPtrField === null || $mapPtrField instanceof CData); + + return Core::mapPtrGet($mapPtrField); + } + + /** + * Mints begin/end trampolines and attaches them to the target function + */ + private function attachHandlers(): void + { + $this->beginSlot = Core::new('zend_observer_fcall_begin_handler[1]'); + $this->beginSlot[0] = Closure::fromCallable([$this, 'handleBegin']); + Core::call('zend_observer_add_begin_handler', $this->function, $this->beginSlot[0]); + + if ($this->endHandler !== null) { + $this->endSlot = Core::new('zend_observer_fcall_end_handler[1]'); + $this->endSlot[0] = Closure::fromCallable([$this, 'handleEnd']); + Core::call('zend_observer_add_end_handler', $this->function, $this->endSlot[0]); + } + } + + /** + * Detaches this hook's begin/end trampolines from the target function + * + * remove_*_handler reports the handler that moved into the removed slot through its out + * parameter; z-engine does not chain observer handlers, so the scratch slot is written and + * discarded. + */ + private function detachHandlers(): void + { + assert($this->beginSlot !== null); + // The out parameters are the arrays decayed to element pointers; reading an element + // value instead would yield PHP null for an empty slot and lose the pointer identity + $nextBegin = Core::new('zend_observer_fcall_begin_handler[1]'); + Core::call( + 'zend_observer_remove_begin_handler', + $this->function, + $this->beginSlot[0], + Core::cast('zend_observer_fcall_begin_handler *', $nextBegin), + ); + + if ($this->endSlot !== null) { + $nextEnd = Core::new('zend_observer_fcall_end_handler[1]'); + Core::call( + 'zend_observer_remove_end_handler', + $this->function, + $this->endSlot[0], + Core::cast('zend_observer_fcall_end_handler *', $nextEnd), + ); + } + } + + /** + * @inheritDoc + * + * The zend_observer add-handler API attaches begin and end handlers separately, so + * HookInterface::handle() is not the dispatch entry point; the engine calls handleBegin() and + * handleEnd() directly. Provided for interface completeness only. + * + * @return never + */ + public function handle(...$rawArguments): never + { + throw new \LogicException('ObserverHook dispatches through handleBegin()/handleEnd(), not handle()'); + } + + /** + * FFI begin callback: void (*)(zend_execute_data *execute_data) + * + * Runs inside the engine while the observed frame is being entered, so it must never let an + * exception escape into C (issue #50): a throw here would cross the FFI boundary as a fatal + * error. Exceptions are contained and downgraded to an E_USER_WARNING. + * + * @param mixed ...$rawArguments Raw C arguments (zend_execute_data*) + */ + public function handleBegin(...$rawArguments): void + { + [$executeData] = $rawArguments; + assert($executeData instanceof CData); + $this->invokeContained('begin', fn() => ($this->beginHandler)(new ExecutionData($executeData))); + } + + /** + * FFI end callback: void (*)(zend_execute_data *execute_data, zval *return_value) + * + * Same containment guarantee as handleBegin(): no exception may cross into the engine. + * + * @param mixed ...$rawArguments Raw C arguments (zend_execute_data*, zval*) + */ + public function handleEnd(...$rawArguments): void + { + [$executeData, $returnValue] = $rawArguments; + assert($executeData instanceof CData); + $endHandler = $this->endHandler; + if ($endHandler === null) { + // Begin-only hook: the engine never had an end trampoline to call + return; + } + $return = ($returnValue instanceof CData) ? ReflectionValue::fromValueEntry($returnValue) : null; + $this->invokeContained('end', static fn() => $endHandler(new ExecutionData($executeData), $return)); + } + + /** + * Invokes a user observer callback with full exception containment + * + * This frame is entered by the engine through an FFI trampoline, so nothing may escape into + * the engine (issue #50): a throw from the callback is downgraded to E_USER_WARNING, and even + * a user error handler converting that warning into an exception is swallowed. + */ + private function invokeContained(string $kind, Closure $callback): void + { + try { + $callback(); + } catch (\Throwable $failure) { + try { + trigger_error( + "Observer {$kind} callback threw " . get_class($failure) . ': ' . $failure->getMessage(), + E_USER_WARNING, + ); + } catch (\Throwable) { + // A user error handler converted the warning into an exception: it must not + // cross the FFI boundary either (issue #50) + } + } + } + + /** + * Best-effort restore for hooks dropped without uninstall() + */ + public function __destruct() + { + if (Core::isShutdown() || !$this->installed) { + return; + } + if (Core::isTopHook($this)) { + $this->uninstall(); + } + } + + /** + * Internal CData fields could result in segfaults, so let's hide everything + * + * @return array + */ + public function __debugInfo(): array + { + return [ + 'installed' => $this->installed, + 'isUserFunction' => $this->isUserFunction, + 'beginHandler' => $this->beginHandler, + 'endHandler' => $this->endHandler, + ]; + } +} diff --git a/tests/Stub/observerFiringProbe.php b/tests/Stub/observerFiringProbe.php new file mode 100644 index 00000000..3af7f410 --- /dev/null +++ b/tests/Stub/observerFiringProbe.php @@ -0,0 +1,173 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +/** + * opcache.preload fixture for ObserverHookFiringTest. + * + * Runs with the observer_enabler test extension loaded (the minimal startup-time observer + * provider), so the engine fcall-observer machinery is ENABLED and the runtime per-function + * observer API is live. Exercises the full ObserverHook firing path and records every event into + * the file named by the ZOBS_OUT environment variable (a plain file: stream resources opened + * during preload break preload finalization). + * + * Scenario selection via ZOBS_SCENARIO: + * - "firing" (default): begin/end + return values, uninstall, nested ordering, + * begin-only observation of a throwing function, callback-exception containment, + * internal-function observation. + * - "throw-with-end": pins the documented hard limitation - an END handler attached to a + * function that throws is invoked by the engine during unwinding, which ext/ffi aborts + * ("Throwing from FFI callbacks is not allowed"). The child process is expected to die. + */ + +use ZEngine\Core; +use ZEngine\Reflection\ReflectionFunction; +use ZEngine\Reflection\ReflectionValue; +use ZEngine\System\ExecutionData; + +require dirname(__DIR__, 2) . '/vendor/autoload.php'; + +Core::preload(); + +$out = getenv('ZOBS_OUT'); +assert(is_string($out) && $out !== ''); +$report = static function (string $line) use ($out): void { + file_put_contents($out, $line . "\n", FILE_APPEND); +}; + +$report('PRELOADED=' . (Core::isPreloaded() ? '1' : '0')); +$report('USER_ENABLED=' . (Core::isObserverEnabled(true) ? '1' : '0')); +$report('INTERNAL_ENABLED=' . (Core::isObserverEnabled(false) ? '1' : '0')); +$report('OBSERVER_COUNT=' . Core::observerFcallObserverCount()); + +// Targets are compiled here - after startup, on the observed side of the boundary +include __DIR__ . '/observerFiringTargets.php'; + +$events = []; +$hookFor = static function (string $name) use (&$events) { + return Core::observeFunction( + (new ReflectionFunction($name))->getRawFunctionPointer(), + static function (ExecutionData $frame) use (&$events, $name): void { + $events[] = "begin:{$name}"; + }, + static function (ExecutionData $frame, ?ReflectionValue $return) use (&$events, $name): void { + $rendered = 'null'; + if ($return !== null) { + $native = null; + $return->getNativeValue($native); + $rendered = var_export($native, true); + } + $events[] = "end:{$name}={$rendered}"; + }, + ); +}; + +if (getenv('ZOBS_SCENARIO') === 'throw-with-end') { + // Deliberately attach an END handler to a throwing function: the engine will invoke the + // FFI end trampoline during unwinding and ext/ffi aborts the process. Pinned by the test. + $hookFor('zengine_observed_thrower'); + $report('THROW_WITH_END=armed'); + try { + zengine_observed_thrower(); + } catch (\RuntimeException $exception) { + // Never reached: ext/ffi aborts before the catch can run + $report('THROW_WITH_END=caught'); + } + $report('THROW_WITH_END=survived'); + + return; +} + +// --- 1. begin/end fire with the return value --------------------------------- +$hook = $hookFor('zengine_observed_simple'); +$result = zengine_observed_simple(21); +$report("SIMPLE_RESULT={$result}"); +$report('SIMPLE_EVENTS=' . implode(',', $events)); + +// --- 2. uninstall detaches cleanly ------------------------------------------- +$hook->uninstall(); +$events = []; +$silent = zengine_observed_simple(5); +$report("AFTER_UNINSTALL_RESULT={$silent}"); +$report('AFTER_UNINSTALL_EVENTS=' . implode(',', $events)); + +// --- 3. nested call ordering -------------------------------------------------- +$events = []; +$outerHook = $hookFor('zengine_observed_outer'); +$innerHook = $hookFor('zengine_observed_inner'); +$nested = zengine_observed_outer(1); +$report("NESTED_RESULT={$nested}"); +$report('NESTED_EVENTS=' . implode(',', $events)); +$innerHook->uninstall(); +$outerHook->uninstall(); + +// --- 4. exception in the observed function (begin-only hook) ------------------ +$events = []; +$throwHook = Core::observeFunction( + (new ReflectionFunction('zengine_observed_thrower'))->getRawFunctionPointer(), + static function (ExecutionData $frame) use (&$events): void { + $events[] = 'begin:zengine_observed_thrower'; + }, +); +try { + zengine_observed_thrower(); +} catch (\RuntimeException $exception) { + $report('THROW_CAUGHT=' . $exception->getMessage()); +} +$report('THROW_EVENTS=' . implode(',', $events)); +$throwHook->uninstall(); + +// --- 5. exception in the begin callback is contained --------------------------- +$events = []; +$warning = ''; +set_error_handler(static function (int $severity, string $message) use (&$warning): bool { + $warning = $message; + + return true; +}, E_USER_WARNING); +$brokenHook = Core::observeFunction( + (new ReflectionFunction('zengine_observed_simple'))->getRawFunctionPointer(), + static function (): void { + throw new \LogicException('callback exploded'); + }, + static function () use (&$events): void { + $events[] = 'end-after-broken-begin'; + }, +); +$contained = zengine_observed_simple(3); +restore_error_handler(); +$report("CONTAINED_RESULT={$contained}"); +$report("CONTAINED_WARNING={$warning}"); +$report('CONTAINED_EVENTS=' . implode(',', $events)); +$brokenHook->uninstall(); + +// --- 6. internal function observation ------------------------------------------ +$events = []; +$internalHook = Core::observeFunction( + (new ReflectionFunction('strrev'))->getRawFunctionPointer(), + static function (ExecutionData $frame) use (&$events): void { + $events[] = 'begin:strrev'; + }, + static function (ExecutionData $frame, ?ReflectionValue $return) use (&$events): void { + $native = null; + if ($return !== null) { + $return->getNativeValue($native); + } + $events[] = 'end:strrev=' . var_export($native, true); + }, +); +$reversed = strrev('abc'); +$report("INTERNAL_RESULT={$reversed}"); +$report('INTERNAL_EVENTS=' . implode(',', $events)); +$internalHook->uninstall(); + +$report('DONE'); diff --git a/tests/Stub/observerFiringTargets.php b/tests/Stub/observerFiringTargets.php new file mode 100644 index 00000000..2f2a59a6 --- /dev/null +++ b/tests/Stub/observerFiringTargets.php @@ -0,0 +1,41 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +/** + * Observed target functions for the observerFiringProbe fixture. + * + * Kept in a separate file (included by the preload probe after Core::preload()) for two reasons: + * the file is compiled AFTER engine startup, i.e. on the observed side of the compile-time + * boundary, and declaring functions directly inside the Core::preload() script breaks preload + * finalization. + */ + +function zengine_observed_simple(int $value): int +{ + return $value * 2; +} + +function zengine_observed_inner(int $value): int +{ + return $value + 1; +} + +function zengine_observed_outer(int $value): int +{ + return zengine_observed_inner($value) + 10; +} + +function zengine_observed_thrower(): void +{ + throw new RuntimeException('observed failure'); +} diff --git a/tests/Stub/observerPreloadProbe.php b/tests/Stub/observerPreloadProbe.php new file mode 100644 index 00000000..502e2aae --- /dev/null +++ b/tests/Stub/observerPreloadProbe.php @@ -0,0 +1,54 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +/** + * opcache.preload fixture for ObserverHookPreloadTest. + * + * Runs during engine startup - the only moment observer registration timing is available - and + * records, into the file named by the ZOBS_OUT environment variable, what the observer machinery + * looks like from the preload path on a stock build with no startup-time observer provider: + * - PRELOADED: whether Core booted through the preload path, + * - OBSERVER_ENABLED: whether the engine fcall-observer machinery is enabled, + * - OBSERVE: whether ObserverHook attached or refused with a typed exception. + * + * Output goes to a plain file rather than a stream: opening a php://stderr resource during preload + * would leave a persistent resource that breaks preload finalization. + */ + +use ZEngine\Core; +use ZEngine\Reflection\ReflectionFunction; +use ZEngine\System\Hook\ObserverException; + +require dirname(__DIR__, 2) . '/vendor/autoload.php'; + +Core::preload(); + +$out = getenv('ZOBS_OUT'); +assert(is_string($out) && $out !== ''); +$report = static function (string $line) use ($out): void { + file_put_contents($out, $line . "\n", FILE_APPEND); +}; + +$report('PRELOADED=' . (Core::isPreloaded() ? '1' : '0')); +$report('OBSERVER_ENABLED=' . (Core::isObserverEnabled() ? '1' : '0')); + +// strlen is a pre-existing internal function, so no user function has to be compiled inside the +// preload script (mixing Core::preload() with user function declarations breaks preloading). +$function = (new ReflectionFunction('strlen'))->getRawFunctionPointer(); + +try { + Core::observeFunction($function, static function (): void {}, static function (): void {}); + $report('OBSERVE=attached'); +} catch (ObserverException $exception) { + $report('OBSERVE=rejected'); +} diff --git a/tests/System/Hook/ObserverHookFiringTest.php b/tests/System/Hook/ObserverHookFiringTest.php new file mode 100644 index 00000000..a9b598d2 --- /dev/null +++ b/tests/System/Hook/ObserverHookFiringTest.php @@ -0,0 +1,243 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace ZEngine\System\Hook; + +use PHPUnit\Framework\TestCase; + +/** + * End-to-end firing path of the observer bridge, verified against a real startup-time observer + * provider: the test compiles the minimal observer_enabler extension + * (tests/fixtures/observer-enabler) on demand with the local PHP toolchain, loads it into a child + * process together with the opcache.preload fixture, and asserts on the event log the fixture + * produces - begin/end firing with return values, clean uninstall, nested-call ordering, + * begin-only observation of throwing functions, callback-exception containment, and + * internal-function observation. + * + * A second child pins the documented hard limitation: an END handler attached to a function that + * throws is invoked by the engine during unwinding, and ext/ffi aborts the process before any PHP + * runs ("Throwing from FFI callbacks is not allowed") - if a future PHP release lifts this, the + * pin fails and the begin-only restriction can be revisited. + * + * Skips cleanly when the toolchain (phpize / cc) or opcache is unavailable, so environments + * without build tools stay green. + */ +final class ObserverHookFiringTest extends TestCase +{ + private static ?string $extensionPath = null; + + public static function setUpBeforeClass(): void + { + if (!extension_loaded('Zend OPcache')) { + self::markTestSkipped('opcache is required to exercise the preload path'); + } + self::$extensionPath = self::buildEnablerExtension(); + } + + public function testObserverHookFiresForUserlandAndInternalFunctions(): void + { + $report = $this->runFiringChild('firing', $exitCode, $stdout, $stderr, $context); + + self::assertSame(0, $exitCode, "Firing child exited abnormally\n{$context}"); + self::assertStringContainsString('REQUEST_OK', $stdout, $context); + + // The provider enabled the machinery before the preload script ran + self::assertStringContainsString('PRELOADED=1', $report, $context); + self::assertStringContainsString('USER_ENABLED=1', $report, $context); + self::assertStringContainsString('INTERNAL_ENABLED=1', $report, $context); + self::assertStringContainsString('OBSERVER_COUNT=1', $report, $context); + + // 1. begin fires on entry, end fires on return and sees the return value + self::assertStringContainsString('SIMPLE_RESULT=42', $report, $context); + self::assertStringContainsString( + 'SIMPLE_EVENTS=begin:zengine_observed_simple,end:zengine_observed_simple=42', + $report, + $context, + ); + + // 2. uninstall detaches cleanly: same function, no further events + self::assertStringContainsString('AFTER_UNINSTALL_RESULT=10', $report, $context); + self::assertStringContainsString("AFTER_UNINSTALL_EVENTS=\n", $report, $context); + + // 3. nested calls: outer-begin, inner-begin, inner-end, outer-end + self::assertStringContainsString('NESTED_RESULT=12', $report, $context); + self::assertStringContainsString( + 'NESTED_EVENTS=begin:zengine_observed_outer,begin:zengine_observed_inner,' + . 'end:zengine_observed_inner=2,end:zengine_observed_outer=12', + $report, + $context, + ); + + // 4. a throwing observed function propagates normally under a begin-only hook + self::assertStringContainsString('THROW_CAUGHT=observed failure', $report, $context); + self::assertStringContainsString('THROW_EVENTS=begin:zengine_observed_thrower', $report, $context); + + // 5. an exception in the begin callback is contained (E_USER_WARNING), execution and + // the end handler continue unharmed + self::assertStringContainsString('CONTAINED_RESULT=6', $report, $context); + self::assertStringContainsString( + 'CONTAINED_WARNING=Observer begin callback threw LogicException: callback exploded', + $report, + $context, + ); + self::assertStringContainsString('CONTAINED_EVENTS=end-after-broken-begin', $report, $context); + + // 6. internal functions are observed through the internal-function extension slot + self::assertStringContainsString('INTERNAL_RESULT=cba', $report, $context); + self::assertStringContainsString("INTERNAL_EVENTS=begin:strrev,end:strrev='cba'", $report, $context); + + self::assertStringContainsString('DONE', $report, $context); + } + + public function testEndHandlerOnThrowingFunctionAbortsPinnedLimitation(): void + { + $report = $this->runFiringChild('throw-with-end', $exitCode, $stdout, $stderr, $context); + + // The hook attached and the call started... + self::assertStringContainsString('THROW_WITH_END=armed', $report, $context); + // ...but ext/ffi aborted the process while the engine unwound the throwing frame: + // the catch block never ran and the process died with the documented fatal error. + self::assertStringNotContainsString('THROW_WITH_END=caught', $report, $context); + self::assertStringNotContainsString('THROW_WITH_END=survived', $report, $context); + self::assertNotSame(0, $exitCode, "Expected the child to abort\n{$context}"); + self::assertStringContainsString('Throwing from FFI callbacks is not allowed', $stdout . $stderr, $context); + } + + /** + * Launches the preload firing fixture in a child process and returns its report + * + * @param-out int $exitCode + * @param-out string $stdout + * @param-out string $stderr + * @param-out string $context + */ + private function runFiringChild( + string $scenario, + ?int &$exitCode, + ?string &$stdout, + ?string &$stderr, + ?string &$context, + ): string { + self::assertIsString(self::$extensionPath); + $fixture = dirname(__DIR__, 2) . '/Stub/observerFiringProbe.php'; + $reportOut = tempnam(sys_get_temp_dir(), 'zobs_'); + self::assertIsString($reportOut); + + $command = [ + PHP_BINARY, + '-d', 'extension=' . self::$extensionPath, + '-d', 'ffi.enable=1', + '-d', 'opcache.enable_cli=1', + '-d', 'opcache.jit=off', + '-d', 'opcache.preload=' . $fixture, + '-r', 'echo "REQUEST_OK\n";', + ]; + + /** @var array $inheritedEnv */ + $inheritedEnv = getenv(); + $environment = ['ZOBS_OUT' => $reportOut, 'ZOBS_SCENARIO' => $scenario] + $inheritedEnv; + $process = proc_open( + $command, + [1 => ['pipe', 'w'], 2 => ['pipe', 'w']], + $pipes, + null, + $environment, + ); + self::assertIsResource($process, 'Unable to spawn the firing child process'); + + $stdout = stream_get_contents($pipes[1]) ?: ''; + $stderr = stream_get_contents($pipes[2]) ?: ''; + fclose($pipes[1]); + fclose($pipes[2]); + $exitCode = proc_close($process); + + $report = (string) file_get_contents($reportOut); + @unlink($reportOut); + $context = "SCENARIO={$scenario} EXIT={$exitCode}\nSTDOUT:\n{$stdout}\nSTDERR:\n{$stderr}\nREPORT:\n{$report}"; + + if (str_contains($stderr, 'Preloading is not supported') || str_contains($stderr, 'preload_user')) { + self::markTestSkipped("Preloading unavailable in this environment:\n{$context}"); + } + + return $report; + } + + /** + * Builds (or reuses) the observer_enabler provider extension with the local PHP toolchain + * + * The build runs in a cached temp directory keyed by source hash and PHP version, so repeated + * test runs reuse the compiled .so. Skips the whole test class when the toolchain is missing. + */ + private static function buildEnablerExtension(): string + { + foreach (['phpize', 'cc', 'make'] as $tool) { + if (self::runCommand(['sh', '-c', "command -v {$tool}"], sys_get_temp_dir()) !== 0) { + self::markTestSkipped("Build tool '{$tool}' is not available"); + } + } + + $sourceDir = dirname(__DIR__, 2) . '/fixtures/observer-enabler'; + $source = $sourceDir . '/observer_enabler.c'; + $configM4 = $sourceDir . '/config.m4'; + $cacheKey = substr(md5(PHP_VERSION_ID . '|' . md5_file($source) . '|' . md5_file($configM4)), 0, 12); + $buildDir = sys_get_temp_dir() . '/z-engine-observer-enabler-' . $cacheKey; + $module = $buildDir . '/modules/observer_enabler.so'; + + if (is_file($module)) { + return $module; + } + + if (!is_dir($buildDir) && !mkdir($buildDir, 0777, true) && !is_dir($buildDir)) { + self::markTestSkipped("Cannot create build directory {$buildDir}"); + } + copy($source, $buildDir . '/observer_enabler.c'); + copy($configM4, $buildDir . '/config.m4'); + + foreach ([['phpize'], ['sh', '-c', './configure --quiet'], ['make', '-s']] as $step) { + if (self::runCommand($step, $buildDir, $output) !== 0) { + self::markTestSkipped('Building observer_enabler failed at "' . implode(' ', $step) . "\":\n{$output}"); + } + } + if (!is_file($module)) { + self::markTestSkipped('observer_enabler build completed but produced no module'); + } + + return $module; + } + + /** + * Runs a build command in a working directory, capturing combined output + * + * @param list $command + * @param-out string $output + */ + private static function runCommand(array $command, string $workingDirectory, ?string &$output = null): int + { + $process = proc_open( + $command, + [1 => ['pipe', 'w'], 2 => ['pipe', 'w']], + $pipes, + $workingDirectory, + ); + if (!is_resource($process)) { + $output = 'proc_open failed'; + + return 1; + } + $output = (stream_get_contents($pipes[1]) ?: '') . (stream_get_contents($pipes[2]) ?: ''); + fclose($pipes[1]); + fclose($pipes[2]); + + return proc_close($process); + } +} diff --git a/tests/System/Hook/ObserverHookPreloadTest.php b/tests/System/Hook/ObserverHookPreloadTest.php new file mode 100644 index 00000000..df1c287c --- /dev/null +++ b/tests/System/Hook/ObserverHookPreloadTest.php @@ -0,0 +1,86 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace ZEngine\System\Hook; + +use PHPUnit\Framework\TestCase; + +/** + * Observer bridge behaviour on the opcache.preload boot path, exercised in a child process because + * observer registration timing only exists during engine startup and cannot be reproduced in the + * PHPUnit worker (which is already past startup). + * + * This pins the observed/unobserved boundary on a stock build: the engine freezes its + * fcall-observer configuration during startup (zend_observer_post_startup) BEFORE the preload + * script runs, so with no startup-time observer provider present the machinery is disabled and + * ObserverHook refuses to attach with a typed exception rather than corrupting memory. See + * docs/observer-hook.md for the full analysis. + */ +final class ObserverHookPreloadTest extends TestCase +{ + public function testPreloadPathReportsDisabledObserversAndRefusesToAttach(): void + { + if (!extension_loaded('Zend OPcache')) { + self::markTestSkipped('opcache is required to exercise the preload path'); + } + + $fixture = dirname(__DIR__, 2) . '/Stub/observerPreloadProbe.php'; + $reportOut = tempnam(sys_get_temp_dir(), 'zobs_'); + self::assertIsString($reportOut); + + $command = [ + PHP_BINARY, + '-d', 'ffi.enable=1', + '-d', 'opcache.enable_cli=1', + '-d', 'opcache.jit=off', + '-d', 'opcache.preload=' . $fixture, + '-r', 'echo "REQUEST_OK\n";', + ]; + + /** @var array $inheritedEnv */ + $inheritedEnv = getenv(); + $environment = ['ZOBS_OUT' => $reportOut] + $inheritedEnv; + $process = proc_open( + $command, + [1 => ['pipe', 'w'], 2 => ['pipe', 'w']], + $pipes, + null, + $environment, + ); + self::assertIsResource($process, 'Unable to spawn the preload child process'); + + $stdout = stream_get_contents($pipes[1]) ?: ''; + $stderr = stream_get_contents($pipes[2]) ?: ''; + fclose($pipes[1]); + fclose($pipes[2]); + $exitCode = proc_close($process); + + $report = (string) file_get_contents($reportOut); + @unlink($reportOut); + $context = "EXIT={$exitCode}\nSTDOUT:\n{$stdout}\nSTDERR:\n{$stderr}\nREPORT:\n{$report}"; + + if (str_contains($stderr, 'Preloading is not supported') || str_contains($stderr, 'preload_user')) { + self::markTestSkipped("Preloading unavailable in this environment:\n{$context}"); + } + + self::assertSame(0, $exitCode, "Preload child exited abnormally\n{$context}"); + self::assertStringContainsString('REQUEST_OK', $stdout, "Request did not run after preload\n{$context}"); + + // The engine booted through the preload path... + self::assertStringContainsString('PRELOADED=1', $report, $context); + // ...but on a stock build (no startup observer provider) the machinery stays disabled... + self::assertStringContainsString('OBSERVER_ENABLED=0', $report, $context); + // ...so ObserverHook refuses to attach instead of writing into unsized structures. + self::assertStringContainsString('OBSERVE=rejected', $report, $context); + } +} diff --git a/tests/System/Hook/ObserverHookTest.php b/tests/System/Hook/ObserverHookTest.php new file mode 100644 index 00000000..bc01870f --- /dev/null +++ b/tests/System/Hook/ObserverHookTest.php @@ -0,0 +1,166 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace ZEngine\System\Hook; + +use ArrayObject; +use FFI\CData; +use PHPUnit\Framework\TestCase; +use ZEngine\Core; +use ZEngine\Reflection\ReflectionFunction; +use ZEngine\System\ExecutionData; + +/** + * Guards and callback containment of the observer bridge that are verifiable without a startup-time + * observer provider (i.e. on a stock build where the engine fcall-observer machinery is disabled). + * + * The end-to-end firing path requires the engine observer machinery to be enabled at startup, which + * userland/FFI cannot arrange; the preload-timing and observed/unobserved boundary are covered by + * ObserverHookPreloadTest, and the memory-safety reasoning is documented in docs/observer-hook.md. + */ +final class ObserverHookTest extends TestCase +{ + /** + * The test bootstrap boots through Core::init(), not Core::preload(), so observer registration + * timing is unavailable: install() must refuse with the typed exception, never silently no-op. + */ + public function testInstallOutsidePreloadIsRejected(): void + { + $function = (new ReflectionFunction('strlen'))->getRawFunctionPointer(); + $hook = new ObserverHook($function, static function (): void {}, static function (): void {}); + + $this->assertFalse(Core::isPreloaded(), 'Test bootstrap must not run through the preload path'); + + $this->expectException(ObserverException::class); + $this->expectExceptionMessage('opcache.preload'); + $hook->install(); + } + + public function testConvenienceEntryPointRejectsOutsidePreload(): void + { + $function = (new ReflectionFunction('strlen'))->getRawFunctionPointer(); + + $this->expectException(ObserverException::class); + Core::observeFunction($function, static function (): void {}, static function (): void {}); + } + + public function testBeginCallbackExceptionIsContained(): void + { + $hook = new ObserverHook( + (new ReflectionFunction('strlen'))->getRawFunctionPointer(), + static function (): void { + throw new \RuntimeException('boom in begin'); + }, + static function (): void {}, + ); + + $warning = $this->captureWarning(static fn() => $hook->handleBegin(self::fakeExecuteData())); + + $this->assertStringContainsString('Observer begin callback threw', $warning); + $this->assertStringContainsString('boom in begin', $warning); + } + + public function testEndCallbackExceptionIsContained(): void + { + $hook = new ObserverHook( + (new ReflectionFunction('strlen'))->getRawFunctionPointer(), + static function (): void {}, + static function (): void { + throw new \RuntimeException('boom in end'); + }, + ); + + // A null return-value pointer is tolerated (generators / abrupt returns pass NULL) + $warning = $this->captureWarning(static fn() => $hook->handleEnd(self::fakeExecuteData(), null)); + + $this->assertStringContainsString('Observer end callback threw', $warning); + $this->assertStringContainsString('boom in end', $warning); + } + + public function testBeginCallbackReceivesExecutionData(): void + { + $seen = new ArrayObject(); + $hook = new ObserverHook( + (new ReflectionFunction('strlen'))->getRawFunctionPointer(), + static function ($frame) use ($seen): void { + $seen->append($frame instanceof ExecutionData ? 'execution-data' : 'other'); + }, + static function (): void {}, + ); + + $hook->handleBegin(self::fakeExecuteData()); + + $this->assertSame(['execution-data'], $seen->getArrayCopy()); + } + + public function testHandleIsNotTheDispatchEntryPoint(): void + { + $hook = new ObserverHook( + (new ReflectionFunction('strlen'))->getRawFunctionPointer(), + static function (): void {}, + static function (): void {}, + ); + + $this->assertFalse($hook->isInstalled()); + $this->assertFalse($hook->hasOriginalHandler()); + + $this->expectException(\LogicException::class); + $hook->handle(); + } + + public function testFieldKeyIsScopedToTheTargetFunction(): void + { + $strlen = (new ReflectionFunction('strlen'))->getRawFunctionPointer(); + $strrev = (new ReflectionFunction('strrev'))->getRawFunctionPointer(); + + $first = new ObserverHook($strlen, static function (): void {}, static function (): void {}); + $second = new ObserverHook($strlen, static function (): void {}, static function (): void {}); + $other = new ObserverHook($strrev, static function (): void {}, static function (): void {}); + + $this->assertStringStartsWith('observer-fcall::', $first->getHookFieldKey()); + $this->assertSame($first->getHookFieldKey(), $second->getHookFieldKey()); + $this->assertNotSame($first->getHookFieldKey(), $other->getHookFieldKey()); + } + + /** + * Runs a callback expected to trigger exactly one E_USER_WARNING and returns its message + */ + private function captureWarning(callable $callback): string + { + $message = ''; + set_error_handler(static function (int $severity, string $text) use (&$message): bool { + $message = $text; + + return true; + }, E_USER_WARNING); + try { + $callback(); + } finally { + restore_error_handler(); + } + $this->assertNotSame('', $message, 'Expected an E_USER_WARNING to be triggered'); + + return $message; + } + + /** + * A throwaway zend_execute_data pointer; ExecutionData only stores it, so the containment and + * scope tests never dereference engine memory through it. + */ + private static function fakeExecuteData(): CData + { + $frame = Core::new('zend_execute_data'); + + return Core::addr($frame); + } +} diff --git a/tests/fixtures/observer-enabler/config.m4 b/tests/fixtures/observer-enabler/config.m4 new file mode 100644 index 00000000..aee1f8e7 --- /dev/null +++ b/tests/fixtures/observer-enabler/config.m4 @@ -0,0 +1,9 @@ +dnl config.m4 for the test-only observer_enabler extension (see observer_enabler.c) +PHP_ARG_ENABLE([observer_enabler], + [whether to enable observer_enabler], + [AS_HELP_STRING([--enable-observer-enabler], [Enable the observer_enabler test extension])], + [yes]) + +if test "$PHP_OBSERVER_ENABLER" != "no"; then + PHP_NEW_EXTENSION(observer_enabler, observer_enabler.c, $ext_shared) +fi diff --git a/tests/fixtures/observer-enabler/observer_enabler.c b/tests/fixtures/observer-enabler/observer_enabler.c new file mode 100644 index 00000000..a19d1012 --- /dev/null +++ b/tests/fixtures/observer-enabler/observer_enabler.c @@ -0,0 +1,52 @@ +/* + * observer_enabler - test-only extension module for z-engine's ObserverHook tests. + * + * Registers an fcall observer during MINIT that observes nothing ({NULL, NULL} + * handlers for every function). Its only purpose is to make the engine enable + * the zend_observer machinery - reserve the op_array / internal-function + * extension slots in zend_observer_post_startup() - so that the runtime + * per-function observer API becomes usable by z-engine's ObserverHook. + * + * This is the minimal "startup-time observer provider" described in + * docs/observer-hook.md. Built on demand by ObserverHookFiringTest via + * phpize / configure / make; never installed permanently. + */ +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif + +#include "php.h" +#include "zend_observer.h" + +static zend_observer_fcall_handlers observer_enabler_init(zend_execute_data *execute_data) +{ + zend_observer_fcall_handlers handlers = {NULL, NULL}; + + (void) execute_data; + + return handlers; +} + +static PHP_MINIT_FUNCTION(observer_enabler) +{ + zend_observer_fcall_register(observer_enabler_init); + + return SUCCESS; +} + +zend_module_entry observer_enabler_module_entry = { + STANDARD_MODULE_HEADER, + "observer_enabler", + NULL, /* functions */ + PHP_MINIT(observer_enabler), + NULL, /* MSHUTDOWN */ + NULL, /* RINIT */ + NULL, /* RSHUTDOWN */ + NULL, /* MINFO */ + "1.0.0", + STANDARD_MODULE_PROPERTIES +}; + +#ifdef COMPILE_DL_OBSERVER_ENABLER +ZEND_GET_MODULE(observer_enabler) +#endif diff --git a/tools/generator/emit.php b/tools/generator/emit.php index 968ffca3..7eacb78c 100644 --- a/tools/generator/emit.php +++ b/tools/generator/emit.php @@ -134,6 +134,8 @@ function runTo(string $command, string $stdoutFile): void #include "zend_modules.h" #include "zend_arena.h" #include "zend_exceptions.h" + #include "zend_extensions.h" + #include "zend_observer.h" #include "supplement.h" C; file_put_contents($buildDir . '/input.c', $inputC); diff --git a/tools/generator/symbols.php b/tools/generator/symbols.php index ac388823..d0dadc02 100644 --- a/tools/generator/symbols.php +++ b/tools/generator/symbols.php @@ -86,6 +86,25 @@ // Opcode API 'zend_set_user_opcode_handler', 'zend_get_user_opcode_handler', + // Observer API (zend_observer.h). Registration (zend_observer_fcall_register) + // is only honoured before zend_observer_post_startup(); the per-function + // add/remove handlers attach begin/end callbacks at runtime. See + // src/System/Hook/ObserverHook.php and docs/observer-hook.md for the timing + // and memory-safety boundary that governs their use from z-engine. + 'zend_observer_fcall_register', + 'zend_observer_add_begin_handler', + 'zend_observer_add_end_handler', + 'zend_observer_remove_begin_handler', + 'zend_observer_remove_end_handler', + // Allocates a user function's run_time_cache without executing it, so the + // observer handler slot can be initialised before the first call. + 'zend_init_func_run_time_cache', + // Total internal-function extension slot bytes (handles * sizeof(void*)). + // The observer block is always the LAST internal-handle reservation + // (zend_observer_post_startup runs after every MINIT), so the number of + // registered fcall observers is derivable as + // (reserved_size/sizeof(void*) - internal observer slot) / 2. + 'zend_internal_run_time_cache_reserved_size', // Inheritance / object API 'zend_do_inheritance_ex', 'zend_objects_new', @@ -140,6 +159,12 @@ 'module_registry', 'std_object_handlers', 'zend_ast_process', + // Observer extension slots. Reading them tells z-engine whether the engine's + // fcall-observer machinery was enabled by a startup-time (MINIT) provider: + // op_array_extension == -1 means observers are disabled (ZEND_OBSERVER_ENABLED + // is false) and no user function reserves an observer handler slot. + 'zend_observer_fcall_op_array_extension', + 'zend_observer_fcall_internal_function_extension', ], 'defines' => [